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Executive  Summary 

The  Army  transformation  initiative,  and  the  drive  toward  lighter,  more  lethal,  and  highly 
deployable  formations,  and  the  parallel  evolution  of  the  battlefield  towards  a  dynamic,  nonlinear, 
and  highly  dispersed  environment,  leads  to  one  clear  conclusion:  intelligent  unmanned  systems  will 
not  just  be  required,  but  indeed  will  be  critical  to  the  success  of  our  future  forces.  The  defense 
community  has  recognized  the  value  of  even  the  first-generation  fielded,  unmanned  systems.  For 
example  Predator,  the  unmanned  aerial  vehicle,  proved  its  value  as  a  sensor  and  weapons  platform 
in  battlefields  as  diverse  as  Bosnia  and  Afghanistan.  This  has  led  to  a  call  for  increased  use  of 
unmanned  systems  from  the  highest  levels  of  our  military  establishment.  President  Bush  stated  at 
The  Citadel  on  December  11,  2001  that 

“We’re  entering  an  era  in  which  unmanned  vehicles  of  all  kinds  will  take  on  greater 
importance — in  space,  on  land,  in  the  air,  and  at  sea.” 

Further  the  Department  of  Defense  (DoD)  2003  budget  submission  quotes  Secretary  Donald 
Rumsfeld: 

“The  growing  use  of  unmanned  aerial  vehicles,  the  effective  utilization  of  real-time 
intelligence,  and  the  coordination  among  special  operations  and  allied  forces  all 
demonstrate  the  cutting  edge  of  what  military  transformation  can  achieve  and  offer  a 
glimpse  of  a  future  transformed  joint  force.” 

The  Army  Science  Board  (ASB)  has  also  recognized  and  acknowledged  the  value  of 
unmanned  systems  in  several  recent  studies  focused  on  Army  transformation,  including  the 
Summer  Studies  in  1999  and  2000.  The  clear  recognition  in  these  studies  was  that  human 
elements  of  the  force  will  require  and  use  unmanned  systems  in  ways  that  involve  close 
interaction.  Further,  because  the  technology  required  for  full  autonomy  will  not  be  available  in  the 
abbreviated  time  frame  for  the  initiation  of  Army  transformation,  efforts  should  focus  on 
controlled,  semiautonomous  operations.  These  two  observations  lead  naturally  to  the  question  of 
how  humans  and  unmanned  systems  interact.  In  this  study,  we  focused  primarily  on  the  issues 
surrounding  the  interactions  of  humans  with  unmanned  ground  systems.  The  ground  environment 
presents  significant  challenges  to  autonomous  systems,  in  large  part  due  to  the  navigation 
requirements  created  by  the  wide  variability  of  the  terrain,  and  the  close  proximity  between  the 
autonomous  entities  and  humans  in  the  environment.  These  challenges  to  automated  systems 
operation  make  the  task  of  effective  human-robot  interactions  particularly  important  to  mission 
success.  Further,  airborne  vehicles  alone  cannot  efficiently  perform  a  large  number  of  tasks  that 
are  important  to  the  Army  necessitating  the  use  of  unmanned  ground  vehicles.  However,  the 
general  conclusions  we  reached  and  the  recommendations  that  follow  are  applicable  to  airborne 
systems  as  well. 

During  our  study,  the  Human-Robot  Interface  panel  and  its  government  advisors  either 
visited  or  were  briefed  on  various  science  and  technology  programs  within  the  Army,  Air  Force, 
Navy,  National  Aeronautics  and  Space  Administration  (NASA),  and  the  Defense  Advanced 
Projects  Agency  (DARPA).  We  examined  these  programs  to  identify  the  research  and 


development  activities  being  carried  out  to  allow  effective  interactions1  between  humans  and 
(semi)autonomous  systems,  and  to  identify  how  effectively  such  systems  were  being  transitioned 
into  operational  use. 

Our  key  findings  provide  significant  reasons  for  optimism  as  well  as  significant  concern. 

First,  we  conclude  that  the  technology  for  autonomous  robotics  has  matured  significantly.  In 
particular,  the  Army  Research  Laboratory  (ARL)  lead  robotics  program,  Demo  III,  has  made 
significant  progress  in  developing  the  perception  technology  essential  to  autonomous  cross¬ 
country  navigation.  While  much  research  and  development  remains,  the  progress  made  in  the  last 
2  years  (since  the  ASB  last  viewed  the  program)  is  remarkable.  Nonetheless,  we  conclude  that 
no  existing  programs  systematically  approach  the  challenges  of  interactions  between  humans  and 
complex  unmanned  systems.  Existing  literature  contains  numerous  examples  that  show  that  the 
lack  of  rigor  in  the  design  of  interactions  and  interfaces  between  humans  and  complex  systems  can 
lead  to  catastrophic  results  (e.g.,  Three  Mile  Island,  the  USS  Vincennes  shoot-down  of  the  Iran 
Air  Airbus).  If  the  human-robot  interaction  issue  is  not  systematically  addressed,  we  are 
concerned  that  similar  catastrophic  problems  could  arise  in  the  application  of  robotic  platforms  in 
the  Army.  This,  in  turn,  would  result  in  severe  setbacks  to  the  induction  of  robotics  into  the  force. 
Finally,  we  observed  that  no  consolidated  programmatic  drivers  are  providing  the  “user  pull”  for 
semiautonomous  and  autonomous  platforms  that  couple  user  needs  directly  with  fundamental 
research.  The  6.1,  6.2,  6.3,  and  6.4  robotics  communities  are  fragmented,  with  no  single  visible 
advocate  or  manager  for  these  technologies  in  the  Army.  We  are  concerned  that  without  such  a 
driver,  there  is  no  motivation  for  hands-on  experimentation  with  evolving  unmanned  systems.  This 
in  turn  will  restrict  the  development  of  effective  interaction  modalities,  tactics,  techniques,  and 
procedures  that  will  delay  the  induction  of  usable  robotic  entities  into  the  force. 

Our  recommendations  are  divided  into  three  parts.  First,  we  recommend  that  the 
requirements  community,  led  by  TRADOC  and  the  schools,  establish  an  operational  architecture 
for  autonomous  robots,  and  validate  the  architecture  through  an  aggressive  program  of  hands-on 
usage  and  experimentation  with  available  robots  in  the  field;  (e.g.,  by  the  Army  National  Guard, 
by  the  Opposing  Force  (OPFOR)  at  the  National  Training  Center).  Second,  we  recommend  the 
creation  of  a  new  systems -oriented  program  for  the  analysis,  understanding,  development,  and 
improvement  of  human-robot  interactions.  We  recommend  that  ARL,  in  cooperation  with 
DARPA  and  other  technology  and  system  developers  be  the  steward  for  such  a  program.  This 
should  facilitate  technology  insights  and  lessons  learned  from  the  field  use  of  robots  and  the  real 
time  feedback  establish  the  baseline  for  future  developments;  a  process  that  should  promote  spiral 
development.  Finally,  we  recommend  that  the  Army  insist  that  the  Future  Combat  System  (FCS) 
Block  1  program  have,  at  a  minimum,  follower  robots  with  a  significant  level  of  autonomy  and 
surveillance  and  reconnaissance  robots  that  can  operate  in  limited  environments — capabilities  that 
can  developed  by  maturing  the  technology  that  exists  today. 


In  this  report,  we  use  the  terms  unmanned  systems  and  robots  interchangeably.  Further,  we  use 
the  term  interface  and  interaction  interchangeably  for  reasons  that  are  described  later  in  the  text. 


1  TERMS  OF  REFERENCE 
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DEPARTMENT  OF  THE  ARMY 
OFFICE  OF  tHE  ASSISTANT  SECRETARY  OF  THE  ARMY 
ACQUISI  TION  LOGISTICS  AND  TECHNOLOGY 
1M  ARMY  PENTAGON 

Washington  0020110.0103 


14  m  m 


Mr.  Michael  Bayer 

Chair,  Army  Science  Board 

2511  Jefferson  Dam  Highway,  Suite  11500 

Arlington,  Virginia  22202 

Dear  Mr.  Bayer 

I  request  that  the  Army  Science  Board  (ASB)  conduct  a  study  on  “Human- 
Rofaotfc  Interface  Issues11  as  a  means  of  addressing  innovative  methods  for 
interacting  with  and  controlling  semi-automated,  and  fully  automated  systems  on 
the  battlefield ,  The  study  should  address,  but  is  not  limited  to,  the  Terms  of 
Reference  (TOR)  described  below.  Appointed  ASS  members  to  this  study  are  to 
consider  the  TOR  as  guidelines  and  may  expand  the  study  to  issues  considered 
important  to  the  study.  Modifications  to  the  TOR  must  be  addressed  with  the 
Chairman  of  the  ASB. 

Background: 

a-  The  desire  for  rapidly  deployable  forces  and  the  resultant  drive  towards 
lightweight  mechanized  fighting  systems  places  a  great  premium  on  survivability 
by  means  other  than  heavy  armor.  An  "ensemble"  of  manned  and  unmanned 
platforms  operating  cooperatively  as  an  infdnmation-network-integrated  team 
could  provide  the  desired  levels  of  survivability.  In  such  a  construct,  the 
unmanned  platforms  could  provide  at  least  the  following  four  functions: 

(1 }  Serve  as  remote  (potentially  expendable)  sensor  assets  that  would 
support  the  situation  awareness  needed  to  keep  the  manned  platforms  out  of 
harm's  way. 

(2)  Semi-automatedflully  automated  platforms  could  serve  as  logistics 
resupply  systems,  reducing  the  number  of  manned  platforms  required  on  the 
battlefield. 

(3)  Unmanned  platforms  could  provide  communications  relay 
capabilities,  to  ensure  connectivity  among  dispersed  forces. 

(4)  The  scope  of  unmanned  devices  may  extend  to  lethal  applications, 
such  as  indirect  fire  support  or  a  "robotic  wingmem* 
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b,  Depending  on  the  role(s)  that  unmanned  systems  play  on  the 
battlefield,  ft  Is  critical  for  soldiers  to  efficiently  interact  with  and/or  command  and 
control  such  automated  systems.  This  requirement  raises  a  wide  variety  gf  issues 
ranging  from  the  human  factors  of  display  systems,  to  cognitive  models  that 
represent  context  and  shared  awareness  for  collaborative  manned/unmanned 
operations,  to  control  devices  that  are  used  to  manipulate  the  robots.  Since 
technology  for  completely  autonomous  robots  la  unlikely  until  the  2020 
timeframe,  soldiers  would  have  to  either  telephone  operate  the  Unmanned  Air 
Vehicle  {UAVs}  and  Unmanned  Ground  Vehicle  {UGVs)r  or  perform  supervisory 
control  (Le.6  the  robots  operate  autonomously  for  routine  parts  of  the  mission  and 
are  telephone  operated  during  critical  mission  phases). 

TOR.  The  study  should  be  guided  by,  but  not  limited  to  the  following  TOR: 

(1)  Examine  Army,  Defense  Advanced  Research  Projects  Agency,  Navy, 
Air  Force  and  National  Aeronautical  Space  Administration  UGV  and  UAV 
research  &  development  efforts  focused  on  human-machine  interfaces, 
command  and  control  of  robots,  and  supervisory  control . 

(2)  Project  technologies  and  capabilities  into  the  2015-2020  timeframe, 
and  assess  technology  voids  that  may  remain. 

(3)  Determine  the  availability  issues  for  applicable  commercial  systems 
and  technologies. 

(4)  Propose  (cost-effective)  options  or  strategies  for  addressing  any 
technology  voids  identified  above. 

Study  Sponsorship.  The  sponsor  for  this  study  will  be  the  Office  of  the  Assistant 
Secretary  of  the  Army,  Acquisition,,  Logistics,  and  Technology. 

Study  Duration.  The  study  shall  be  completed  by  November  2001. 


Sincerely, 


Kenneth  J.  uscar 

Acting  Assistant  Secretary  of  the  Army 
(Acquisition,  Logistics  and  Technology) 
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2  INTRODUCTION 


Study  Sponsorship  and  Participants 


SPONSOR:  ASAALT 


Task  Force  Members 

Executive  Secretary 

Dr.  Patrick  Eicker 

Dr.  Prasanna  Mulgaonkar  (Chair) 

Dr.  Herbert  Dobbs  (Co-Chair) 

Government  Advisors 

Mr.  Jack  Taylor  (OSD) 

Dr.  John  Blair 

Prof.  Ka  C  Cheok 

Dr.  Rene  Dupontbriand  (ARL) 

Mr.  Bob  Dodd 

ASB  Secretariat  Representative 

Dr.  Mark  Hofmann 

MAJ  Robert  Grier 

Mr.  David  Martinez 

Other  participants 

Dr.  Christine  Mitchell  (*) 

Dr.  Greg  Canavan 

Dr.  Robbi  Perna 

Dr.  Michael  Krause 

(*)  Member  AFSAB 


•  Government  advisors  specifically  selected  to  ensure: 

Views  from  stakeholder  DoD  organizations  carefully  considered  in  study 
Deliberations  of  panel  coordinated  with  parent  organizations 


Figure  1.  Participants 
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Table  1.  Meeting  Schedule 


Location 

Program(s)  Reviewed 

Date 

Study  Panel  Attendees 

Washington  DC 

•  Pat  Eicker  — Robotics  Roadmap 

•  Weatherington  (OSD-ATL)  — UAV 

•  Deitchman  (ONR) — Navy  perspective 

8/31/2001 

Blair,  Hofmann,  Mulgaonkar, 
Dobbs,  Mitchell,  Krause,  Perna 

Ft  Knox 

•  LTC  Abbott  — Battle  Lab 

9/27- 

9/28/01 

Mulgaonkar,  Dodd,  Eicker,  Blair, 
Perna,  Hoffman,  Martinez, 

Cheok,  Dobbs,  Mitchell 

T.  Indiantown 
Gap,  PA 

•  ARL  Demo  III  XUV  demonstration 

Week  of 
11/12/01 

Blair,  Mulgaonkar,  Mitchell, 

Dobbs,  Hoffmann,  Taylor 

Ft  Leonard 
Wood- 

Maneuver 

Support  Center 

•  Chem  school 

•  MP  school 

•  Mini-flail  demo 

11/15/01 

Blair,  Dodd,  Eicker,  Hofmann 

Huntsville 

•  Ground  robots 

•  Software  engineering 

•  System  simulation 

•  Comm  and  others 

11/29- 

30/01 

Mulgaonkar,  Dodd,  Eicker,  Blair, 
Perna,  Hofmann,  Martinez, 

Cheok,  Dobbs,  Mitchell 

DARPA 

At  SRI 

< 

Arlington 

r 

V. 

•  Scott  Fish  -  Perceptor,  UGCV 

•  Doug  Gage,  Jean  Scholtz  -  HRI 

•  Alan  Rudolph 

•  Sam  Wilson  -  MAV 

•  Mike  Leahy  -  UCAV 

•  John  Bay 

12/14/01 

Mulgaonkar,  Dodd,  Eicker,  Blair, 
Mitchell,  Hofmann,  Dobbs 

NIST 

Jim  Albus  -  robot  architectures 

12/14/01 

Mulgaonkar,  Dodd,  Eicker,  Blair, 
Mitchell,  Hofmann,  Dobbs 

Natick  Soldier 
Systems  Center 

Robotics  for  Objective  Force  Warrior 
Workshop 

12/18- 

19/01 

Blair,  Eicker 

SRI 

International 

Greg  Myers — human  robot  interface  issues; 
speech  recognition,  CommandTalk™ 

02/05/02 

Mulgaonkar,  Dodd,  Blair,  Perna, 
Hofmann,  Martinez 

NASA  Ames 

A.  Vera  &  M  Matessa — collaboration 
research  at  Ames 

02/05/02 

Mulgaonkar,  Dodd,  Blair,  Perna, 
Hofmann,  Martinez 

Jet  Propulsion 

Lab 

HRI,  cooperative  behavior  of  robots,  smart 
robot  nav,  robot  software  architecture,  mod- 
sim 

02/06/02 

Mulgaonkar,  Dodd,  Eicker,  Blair, 
Perna,  Hofmann,  Martinez, 

Cheok,  Dobbs 

Sandia  National 
Labs 

UGV  programs,  physics-based  mod-sim, 
remote  manipulation  of  hazardous  objects 

02/07/02 

Mulgaonkar,  Dodd,  Eicker,  Blair, 
Perna,  Hofmann,  Martinez, 

Cheok,  Dobbs 

TACOM 

Jeff  Jaczkowski  —  Vetronics  Overview, 

Crew  Integration  &  Automation  Testbed 
ATD,  Robotic  Follower  ATD  &  Vetronics 
Technology  Testbed  Demo 

3/28/02 

Mulgaonkar,  Blair,  Dobbs, 

Eicker,  Cheok,  Mitchell 

6 


3  ROBOTICS  AND  ITS  PLACE  IN  TRANSFORMATION 


One  might  ask,  “why  is  the  military  interested  in  unmanned  vehicles?”  The  obvious 
motivation  for  this  interest  is  the  benefit  of  using  unmanned  vehicles  to  keep  soldiers  out  of 
harm’s  way  and  increase  soldiers’  effectiveness  by  providing  access  to  areas  that  are  otherwise 
inaccessible.  A  secondary  benefit  is  the  reduction  in  the  weight  of  the  unmanned  platforms 
because  of  the  reduced  need  to  protect  an  occupant  (e.g.,  by  armor  or  environmental  controls). 
Given  these  general  military  objectives,  unmanned  vehicles  should  have  several  general 
capabilities:  to 

1.  Go  somewhere 

2.  Do  something 

3.  Report  to  someone  or  something 

These  capabilities  correspond  to  the  basic  and  enduring  requirements  that  soldiers  be  able  to 
move,  shoot,  and  communicate. 

They  also  imply  that  unmanned  vehicles  include: 

1.  Locomotion  and  navigation  systems — to  get  somewhere 

2.  Mission  equipment  packages — to  do  something 

3.  Communication  Links — to  report  to  someone  or  something. 

What  is  meant  by  locomotion  and  navigation  in  this  context?  Does  it  mean  the  ability  to 
negotiate  all  types  of  terrain  in  all  types  of  weather?  Does  it  mean  that  continuous  interaction 
with  a  soldier  is  required  for  locomotion  control  and  navigation,  or  does  mean  that  less  is  required 
of  the  soldier? 

What  does  a  mission  equipment  package  mean  in  this  context?  Does  it  mean  that  the  vehicle 
carries  sensors  only,  or  weapons  as  well  as  sensors?  How  much  must  the  soldier  interact  with  the 
vehicle  to  use  the  mission  equipment?  Will  different  operators  be  needed  for  the  unmanned 
system  and  the  mission  equipment?  Do  vehicles  accept  different  mission  packages,  depending  on 
the  mission? 

What  is  meant  by  communication  links  in  this  context?  Do  communication  links  connect  the 
soldier  to  one  vehicle  or  to  multiple  vehicles?  Do  communication  links  connect  multiple  vehicles 
for  intervehicle  collaboration?  What  types  and  amounts  of  communication  flow  over  the  links, 
and  how  many  interactions  take  place  between  soldiers? 

Lastly,  to  what  extent  should  we  permit  unmanned  vehicles  do  the  above  things  without 
humans  in  the  loop?  What  should  they  do  by  themselves  or  in  combination  with  other  unmanned 
vehicles,  and  what  should  be  the  role  of  soldiers?  Answers  to  these  questions  will  profoundly 
influence  the  design  of  unmanned  vehicles  as  well  as  the  force  structures  that  will  operate  and 
maintain  them. 

To  understand  why  unmanned  systems  are  a  critical  enabler  for  the  Objective  Force2,  it  is 
instructive  to  examine  a  few  relevant  scenarios.  In  this  section,  we  briefly  discuss  three  aspects  of 
unmanned  systems,  based  on  an  examination  of  three  sources.  The  first  aspect  is  the  “ensemble” 


2  The  Objective  Force  is  the  conceptual  Army  structure  of  the  2020  era. 
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concept  articulated  in  many  sources  but  brought  to  the  forefront  in  the  ASB  1999  Summer  Study. 
The  second  is  a  blend  of  the  operational  concepts  under  discussion  by  the  current  FCS 
community,  including  sources  such  as  ARL,  the  FCS  Lead  Systems  Integrator  (LSI),  and  the  U.S. 
Army  Tank- Automotive  and  Armaments  Command  (TACOM)  Vetronics  Technology  Integration 
(VTI)  communities.  The  third  is  the  long-term  operational  concepts  that  set  the  roadmap  for  the 
tight  integration  of  robots  and  soldiers  into  organic  teams  that  fight  together  instead  of  soldiers 
who  use  the  robots  merely  as  tools  with  which  they  fight. 

1999  Summer  Study  Vision 

In  the  1999  Summer  Study,  one  of  the  key  technology  recommendations  for  producing  a 
lighter  fighting  vehicle  based  force  with  the  lethality  and  survivability  of  today’s  heavy  ground 
platforms  was  the  concept  of  an  ensemble  of  manned  and  unmanned  systems  (air  and  ground).  An 
ensemble  is  a  notional  way  to  deconstruct  the  functions  of  a  heavy  platform  (lethality, 
survivability,  mobility,  etc.)  into  individual  elements  that  could  physically  disperse  into 
components  of  less  than  20  tons  each.  This  concept  is  similar  to  the  way  in  which  a  Navy  battle 
group  uses  specialized  components  to  perform  specialized  functions  (carrier,  anti-submarine 
screen,  etc.)  Figure  2  is  a  graphic  representation  of  this  concept. 


•  Autonomous  robotics  is  integral  to  the  FCS/Objective  force  concept 

•  Interactions  between  humans  and  robots  operating  as  teams  is  critical 


Robotics  TF 
Page  8 


Figure  2.  Ensemble  Vision 

The  success  of  such  a  scheme  requires  that  unmanned  systems  constitute  the  majority  of  the 
ensemble  components.  In  addition,  the  smaller  size  of  the  crew  controlling  an  ensemble 
(compared  to  the  crews  that  control  current  platforms)  will  require  ensemble  components  that  are 
essentially  autonomous.  Note  also  that  the  ensemble  consists  of  autonomous  air  and  ground 
platforms  that  operate  together  in  concert  with  the  manned  elements  of  the  force.  The  concept 
articulated  in  the  summer  study  was  that  ensemble  components  be  networked  into  a  common 
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C4ISR3  infrastructure,  such  that  they  would  provide  electronic  sensor  shields  around  the 
ensemble.  The  ensemble  elements  would  have  both  ground  and  air  mobility  and  would  move  and 
fight  as  a  collective  entity  to  achieve  the  mission  characteristics  required  by  the  Force 
Commander. 

The  FCS  vision 

The  vision  of  the  unmanned  system  for  the  FCS  that  is  the  stepping-stone  for  the  Objective 
Force  is  still  evolving.  Different  organizations  within  the  Army  and  its  contractor  base  have 
developed  concepts  in  which  robotic  platforms  are  used  with  a  variety  of  different  capabilities. 
Figure  3,  a  graphic  from  the  FCS  LSI  envisions  the  use  of  a  number  of  small  (5  T)  reconnaissance 
platforms  similar  to  the  ARL  Demo  III  XUV,  robotic  mules  for  providing  logistical  support  to 
dismounted  units,  and  air  vehicles  of  various  sizes.  While  the  various  scenarios  may  have  different 
details,  they  share  several  common  features.  They  all  postulate  the  use  of  both  ground  and  air 
vehicles;  they  postulate  the  organic  nature  of  the  unmanned  entities  (i.e.,  control  and  ownership  at 
lower  echelons,  even  down  to  the  squad);  and  they  use  robots  as  a  screen  around  the  manned 
entities.  Finally,  they  postulate  a  lethal  capability  for  the  robots  (i.e.,  they  do  more  than 
reconnaissance,  surveillance,  and  target  acquisition  [RSTA]  missions).  On  the  other  hand,  they 
differ  in  their  assumptions  about  the  robustness  and  availability  of  robotic  capabilities. 


3  C4ISR:  Command,  control,  communications,  and  computers,  intelligence,  surveillance,  and 
reconnaissance. 
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Figure  3.  The  FCS  is  envisioned  as  having  several  robotic  platforms 


Given  the  uncertain  availability  of  autonomy  technology,  and  with  a  view  to  accelerate  the 
adoption  and  use  of  unmanned  systems  in  the  Army,  the  ASB  2000  Summer  Study  recommended 
that  the  Army  focus  on  semiautonomous  robots  for  logistics.  For  example,  in  a  convoy  operation, 
humans  could  compensate  for  the  limitation  of  today’s  robots.  The  TACOM  Robotic  Follower 
program  that  adopted  similar  goals  has  developed  scenarios  where  human  operators  use  a 
combination  of  inputs  to  define  the  path  that  the  robotic  vehicles  would  follow.  Figure  4 
graphically  shows  the  TACOM  vision,  and  their  technical  challenges. 
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Figure  4.  Goals  of  TACOM  Robotic  Follower  Program 


Objective  Vision 

We  believe  that  a  greater  integration  of  humans  and  robots  will  characterize  the  battlefield  in 
the  Objective  Force  era  (2020  and  beyond).  We  anticipate  that  robots  will  operate  in  a  team, 
where  each  team  member  (whether  human  or  robotic)  takes  appropriate  initiatives  as 
opportunities  present  themselves.  To  this  end,  robots  not  only  must  have  autonomy  and  reasoning 
capabilities  about  events  in  the  world,  but  they  also  must  communicate  with  each  other  and  with 
their  human  teammates  in  ways  that  are  natural  for  humans.  No  graphic  depictions  of  such 
scenarios  exist  outside  the  realm  of  science  fiction  and  movies  such  as  the  “Star  Wars”  series. 
However,  these  fictional  scenarios  do  indicate  that  focusing  on  the  natural  modes  of  human- 
robotic  interactions  will  be  a  key  driver  to  the  development  of  both  autonomy  and  interfaces. 
Interactions  vs.  Interfaces 

The  terms  of  reference  for  this  study  use  the  word  “interfaces”  to  define  the  subject  of  the 
study.  The  usage  rationale  for  this  derives  from  prior  ASB  observations  that  humans  will  control 
robots  for  the  foreseeable  future.  Consequently,  interfaces  that  humans  will  use  to  perform  these 
control  functions  must  be  developed. 

Early  in  our  study,  it  became  clear  that  the  problem  we  were  asked  to  study  was  actually 
broader  than  interfaces.  As  the  scenarios  just  indicate,  the  character  of  the  interaction  between  the 
manned  and  unmanned  systems  changes.  In  an  ensemble  interaction,  the  unmanned  components 
are  tasked  by  the  manned  components  with  which  they  are  associated.  In  the  FCS  scenarios, 
soldiers  and  commanders  interact  with  the  unmanned  systems  by  issuing  tasks  (i.e.,  RSTA 
missions)  that  support  the  force  commander’s  guidance.  In  the  TACOM  Robotic  Follower  vision, 
humans  interact  with  unmanned  systems  by  simply  performing  a  component  of  the  task;  the 
unmanned  systems  then  “follow.”  In  a  Star  Wars  scenario,  humans  and  robots  interact  in  a  fluid 
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manner,  each  observing  and  responding  to  the  actions,  behaviors,  and  outcomes  of  prior  actions 
(regardless  of  whether  the  actor  is  human  or  robotic). 


The  key  insight  resulting  from  the  panel’s  deliberations  was  that  one  must  first  define  the 
interactions,  and  then  build  interfaces  that  (optimally)  enable  the  desired  interactions  to  take  place. 
In  fact,  one  must  understand  the  overlap  between  the  environments  in  which  a  robot  operates,  the 
robot  missions,  and  the  robot’s  capabilities.  It  is  the  interplay  between  these  elements  that  defines 
the  interactions  that  must  occur  between  the  humans  and  the  robots.  The  interactions,  in  turn, 
drive  the  interfaces.  This  interplay  is  showed  schematically  in  Figure  5.  Other  fields  that 
successfully  demonstrate  this  interaction  definition  approach  for  human-complex  automation 
systems  include  aircraft  flight  decks,  air  traffic  control  and  nuclear  power  plants.  The  lessons 
learned  from  these  environments  are  critical  to  the  unmanned  systems  community,  but,  as  our 
findings  show,  are  not  currently  understood  or  applied. 


In  the  balance  of  this  report,  we  will  preferentially  use  the  term  interaction  except  in  the 
specific  cases  where  we  discuss  the  particular  interfaces  used  to  embody  an  interaction. 
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Figure  5.  Interactions  drive  interfaces 

Status  of  Autonomous  Systems 

From  the  previous  discussion  we  can  see  that  a  critical  driver  for  human-robot  interactions  is 
the  level  of  autonomy.  Clearly,  autonomy  will  improve  over  time,  but  it  is  useful  to  evaluate  its 
current  status. 

Assessing  the  maturity  of  autonomy  technologies  is  subjective,  especially  when  applied  to 
ground  robots.  Further,  autonomy  in  air  vehicles  is  different  from  autonomy  in  ground  vehicles. 
One  of  the  key  research  areas  in  the  field  of  autonomous  unmanned  ground  systems  has  been  the 
development  of  a  type  of  mobility  often  called  A -B  autonomy. 
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Figure  6.  Relationship  of  Unmanned  Ground  Vehicle  (UGV)  Tasks  to  Autonomy. 


Some  assert  that  A-B  autonomy  for  unmanned  ground  vehicles  (UGVs)  rests  in  large 
measure  on  mission  distance  and  velocity  requirements.  From  the  chart  in  Figure  3,  one  can  see 
that  many  useful  functions  can  be  and  are  being  performed  by  unmanned  vehicles  without  A-B 
autonomous  capability. 4  On  the  other  hand,  some  missions  might  well  benefit  from  this 
capability.  It  is  clear  that  many  of  the  technologies  needed  for  a  full  spectrum  of  unmanned 
vehicles  are  in  place.  None  of  these  technologies  are  “show  stoppers”  per  se.  We  believe  that 
there  are  no  programs  in  place  that  have  a  high  confidence  for  developing  A-B  autonomy.  The 
fundamental  thesis  underlying  the  Army’s  A-B  autonomy  program,  Demo  III,  is  the  experimental 
evidence  that  shows  the  inability  to  teleoperate  semiautonomous  UGVs  cross  country  at  high 
speeds.  There  are  two  key  phrases  in  the  latter  sentence:  cross  country  and  high  speed.  If  Force 
Capabilities  exist  that  do  not  require  cross-country  or  high-speed  teleoperation,  then  the  A-B 
autonomy  requirement  may  not  exist.  The  axes  on  the  chart  in  Figure  6  are  distance  ( traveled 
cross-country )  to  mission  point  and  average  velocity.  We  made  a  rough  estimate  of  where  each  of 
the  Force  Capabilities  falls  on  the  chart,  then  coded  the  Force  Capabilities  with  our  estimate  of 
their  overall  technological  readiness.  Some  comments: 


4  See  the  Army  UGV  roadmap,  attached  to  this  document  as  Appendix  C.  Section  1.1.4  is 
particularly  relevant  to  the  point  made  here. 
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•  Many  capabilities  exist.  We  estimate  that  all  the  needed  component  technologies 
are  at  a  sufficient  level  of  technical  readiness  that  only  the  will  remains  to  develop 
and  deploy  such  capabilities 

•  A  number  of  Force  Capabilities  do  not  require  A-B  autonomy. 

•  We  estimate  that  the  capabilities  on  the  top  right  of  the  chart  do  require  A-B 
autonomy.  The  Army  Science  and  Technology  (S&T)  community  must  decide 
whether  these  capabilities  are  critical  to  the  success  of  FCS 

•  The  shaded  region  in  the  middle  is  a  zone  on  one  side  of  which  A-B-  autonomy  is 
clearly  required,  and  on  the  other  side  of  which  teleoperation  of  some  sort  is 
acceptable.5 
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Figure  7.  Estimated  Maturity  of  Autonomous  Land  Navigation 

Figure  7  shows  our  estimate  of  the  maturity  of  the  autonomous  land  navigation  capability 
demonstrated  by  and  extrapolated  from  the  ARL  Demo  III  program.  It  shows  that  under  certain 
bounded  conditions,  autonomous  operations  are  feasible.  For  example,  we  believe  that 
autonomous  cross  country  surveillance  could  be  carried  out  by  XUV  type  robots  over  rolling 
terrain,  using  the  technology  available  today.  Similarly,  robotic  followers  could  be  built  in  a  robust 
fashion  today. 

While  additional  research  in  robotic  autonomy  is  clearly  required,  we  believe  that  an 
appropriate  focus  and  investment  in  maturing  the  available  technology  is  not  only  justified  but 
critical  to  the  success  of  transformation  efforts. 


5  We  recognized  the  need  for  a  similar  matrix  that  addresses  air  vehicles,  but  time  constraints  for 
the  study  prevented  its  development. 
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It  is  important  to  ask  whether  any  of  the  functions  shown  in  Figure  7  would  be  better 
performed  by  aerial  vehicles  instead  of  ground  vehicles.  Table  2  shows  the  potential  distribution 
of  functions  between  UAVs  and  UGVs.  For  example,  engineering  and  construction  must  be 
performed  on  the  ground.  On  the  other  hand,  indirect  fire  can  be  provided  by  air  vehicles,  as  has 
already  been  demonstrated  in  Afghanistan  by  Predator  UAVs  carrying  Hellfire  missiles. 


Table  2.  UAV  and  UGV  Functionality. 


Shaded  areas  represent  tasks  that  coul 

Id  be  performed  by  respective  vehicles. 

Tasks  (from  Figure  6) 

UGV 

Low-Flying  UAV 

High-Flying  UAV 

Logistics  Delivery 

Mule 

Engineering/Construction 

Obstacle  Breaching 

Physical  Security 

Material  Handling 

Nonlethal 

Cave  Clearing 

EOD/UXO* 

(Detection  only) 

Medevac 

Countersniping 

Countermining 

(Detection  only) 

MOUT+Reconnaissance 

NBC*  Detection 

RSTA/BDA 

Direct  Fire 

Indirect  Fire 

*EOD:  Explosive  ordnance  disposal;  UXO:  unexploded  ordnance. 

+  MOUT :  military  operations  in  urban  terrain 
*NBC:  nuclear,  biological,  and  chemical 

The  robotics  community  clearly  recognizes  that  the  challenges  to  autonomy  for  UAVs  are 
different  (and  some  would  argue,  simpler)  from  those  for  UGVs.  What  is  not  widely  recognized  in 
the  community,  is  that  even  when  certain  functions  could  be  assigned  to  UAVs,  high-flying  UAVs 
differ  from  those  that  operate  close  to  the  ground.  Low-flying  UAVs  must  contend  with 
challenges  such  as  trees,  air  currents,  buildings,  dirt  and  flying  debris,  which  might  make 
autonomous  versions  as  difficult  to  build  as  autonomous  UGVs.  In  addition,  UAVs  expend 
significant  energy  to  lift  their  own  weight  and  that  of  their  payloads,  so  they  are  inefficient  in 
applications  that  require  long  periods  of  stationary  operations  and  those  that  require  them  to 
move  significant  weight. 

We  believe  that  the  FCS  LSI  should  focus  effort  on  understanding  this  trade  space  and  create 
a  roadmap  for  the  optimal  distribution  of  functionality  among  various  classes  of  autonomous  and 
semiautonomous  platforms. 
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Figure  8.  Levels  of  Autonomous  Behavior 

As  noted  above,  we  must  understand  what  a  robot  can  do  if  we  are  to  define  human 
interactions  that  are  required.  In  order  to  add  some  rigor  to  the  discussions,  we  defined  levels  of 
autonomy  in  order  to  parameterize  robot  capability.  These  levels  are  shown  in  Figure  8  and  listed 
below. 

0.  Manual  Remote  Control,  like  a  remote  controlled  toy 

1.  Simple  Automation 

2.  Automated  Tasks  and  Functions,  like  a  Hunter 

3.  Scripted  Mission,  like  an  Shadow  or  Predator  UAV 

4.  Semiautomated  Missions  with  Simple  Decision  Making,  like  an  Cruise  Missile 

5.  Complex  Missions-Specific  Reasoning 

6.  Dynamically  Mission  Adaptable 

7.  Synergistic  Multimission  Reasoning 

8.  Human-like  Autonomy  in  a  Mixed  Team 

9.  Autonomous  Teams  with  Unmanned  Leader  or  Mission  Manager 

10.  Autonomous  Conglomerate. 


16 


In  the  UGV  world,  fielded  systems  such  as  the  Panther  and  the  Mini-Flail  are  at  Level  0.  They  are 
simple  remote-controlled  devices  in  which  the  operator  visualizes  the  actions  of  the  robot  through 
direct  observation,  and  controls  the  robot  through  low-level  commands.  Demo  Ill’s  XUV  is  at 
Level  3  or  4,  where  the  robot  performs  a  significant  amount  of  sensor-based,  self-directed 
operation.  There  are  no  extant  systems  at  Levels  9  and  10,  which  represent  only  vague  concepts 
more  akin  to  the  “droids”  of  science  fiction  than  any  real  system  under  development  today. 

4  SOLDIER  INTERACTIONS  AND  INTERFACES  WITH  UNMANNED 

SYSTEMS 

Once  we  accept  the  notion  that  interaction  precedes  interfaces,  it  is  useful  to  step  back  from 
the  technology  and  look  at  how  humans  interact  with  (or  use)  “things” — tools,  weapons,  other 
people  in  general. 

Considered  from  that  viewpoint,  robots  are  simply  a  class  of  things  with  potential  capabilities 
ranging  from  those  of  ordinary  power  tools  to  those  of  humans  and  possibly  beyond.  This 
spectrum  of  potentials  confuses  the  discussion.  A  robot  is  not  a  clearly  defined  thing  like  a  rifle  or 
an  entrenching  tool.  A  robot’s  use  depends  upon  its  intended  function  and  capabilities. 

In  the  near-  and  mid-term,  humans  will  closely  control  most  military  robots  (ASB  Summer 
Study  2001).  The  robot’s  ability  to  perform  autonomously  in  the  unstructured  military 
environment  will  still  be  limited.  Systems  like  the  teleoperated  Panther  minefield  proofing  unit 
used  in  Bosnia  will  be  relatively  common.  However,  the  ARL  Demo  III  RSTA  robot  already  has 
demonstrated  significant  ability  to  go  where  it  is  told  in  the  field,  gather  information,  and  find  its 
way  home  without  further  human  control.  Using  a  robot  like  the  Panther  differs  only  marginally 
from  driving  a  vehicle.  Using  the  Demo  III  robot  would  leave  a  soldier  free  to  do  other  tasks 
while  the  robot  carried  out  its  own  mission. 

The  performance  of  the  Demo  III  robot  is  quite  amazing  to  those  of  us  who  have  watched  the 
program  progress  over  the  past  20  years.  The  next  20  years  predictably  will  show  far  greater 
success.  The  concept  of  a  robotic  mule  (RoboMule)  for  the  Objective  Force  soldier  was  clearly 
supported  by  the  ASB  2001  Summer  Study.  In  the  far  term,  after  2010,  such  RoboMules  will 
have  far  more  capability  than  the  Army  leadership  now  believes  possible. 

RoboMules  will  not  require  self-awareness  but  will  require  enough  intelligence  to  know  the 
soldiers  to  whom  they  are  assigned  and  be  able  to  communicate  verbally  with  them.  They  also 
will  need  to  communicate  electronically  with  other  machines  and  with  soldiers.  They  will  need  to 
find  their  way  to  where  they  are  told  to  go  and  to  take  care  of  their  own  refueling.  They  must 
know  enough  to  ask  permission  before  leaving  their  assigned  soldiers  to  get  fuel  or  pick  up 
routine  supplies  from  some  other  area.  They  should  have  more  capable  sensors  than  a  human  and 
the  ability  to  travel  at  least  50  mph  on  improved  roads.  Off-road  capabilities  should  include  a 
travel  speed  of  20  mph,  60%  forward  and  side  slope  negotiation,  and  the  ability  to  cross  a  ditch  4 
feet  wide.  In  addition,  they  should  have  the  intelligence  necessary  to  recognize  impassable  terrain 
at  a  level  at  least  similar  to  human  ability. 

A  RoboMule  will  require  arms — two  or  more — for  loading  and  unloading  itself  or  other 
vehicles  (and  possibly  other  tasks).  It  will  require  all-around  awareness,  despite  the  visual 
obstruction  of  its  cargo  area;  naturally,  it  will  not  be  limited  to  one  pair  of  eyes  like  a  human.  It 
will  need  the  ability  to  open  and  close  the  top  of  its  cargo  area  for  loading  and  unloading  as  well 


17 


as  to  help  with  the  wounded,  and  safely  load  and  unload  litters. 

The  RoboMule  is  on  the  other  end  of  the  robot  spectrum  from  the  “dumb”  Panther.  The 
basic  point  here  is  that  soldiers  will  interact  with  the  semi-intelligent  RoboMule  much  as  they  do 
with  other  soldiers.  One  of  the  more  challenging  tasks  that  a  human  performs  when  interacting 
with  robots  is  that  of  analyzing  and  reacting  to  information.  The  Rotocraft  Pilot’s  Associate 
(RPA)  demonstrates  how  to  automate  such  tasks  successfully.  In  the  RPA  program,  automated 
decision  tools  coupled  with  mission  execution  and  planning  modules  reduce  the  cognitive 
workload  for  humans.  The  RPA  system  has  allowed  AH-64D  Apache  helicopter  pilots  to  control 
another  UAV  (Hunter)  in  addition  to  their  own  helicopters  in  real  time  and  in  a  real  operating 
environment  (Fort  Polk  JCF-AWE,  and  at  the  NTC  with  the  101st  Air  Assault  Division). 

In  sum,  soldiers  will  use  robots  in  many  different  ways,  just  as  they  use  the  tools,  weapons, 
and  fellow  soldiers  with  whom  they  work  today.  The  task  and  the  tools  at  hand,  whether  robots 
or  ratchet  wrenches,  will  determine  how  the  soldier  uses  them.  Unless  robots  can  perform  tasks 
better  than  humans,  and  the  resulting  human-robotic  combination  produces  a  more  capable 
fighting  force,  there  is  no  value  in  fielding  them.  Initially,  the  human-robotic  interface  will  likely 
involve  simple  jobs  not  significantly  different  from  those  accomplished  with  current  military 
vehicles.  As  robotic  capabilities  increase,  soldiers  will  in  general  give  robots  direction  rather  than 
control  them  directly.  Ultimately,  soldiers  will  work  with  robots  in  the  same  way  as  they  do  with 
other  soldiers. 


Figure  9.  Impact  of  Autonomy  Levels  on  Workload 


It  is  important  to  understand  the  way  in  which  assumptions  about  autonomy  and  robot 
system  capability  factor  into  the  design  of  human  robot  interfaces. 

As  Figure  9  shows,  the  requirement  for  robots  to  execute  certain  functions  autonomously 
varies.  Even  when  autonomy  may  be  required,  the  environment  may  be  complex  enough  that  full 
autonomy  is  not  achievable.  Further,  in  complex  environments  with  semiautonomous  robots,  the 


18 


level  of  supervision  that  is  required  may  actually  put  a  heavier  cognitive  load  on  the  human  than 
simpler  environments  or  simpler  robots. 

In  a  systems-engineering  approach  to  designing  complex  systems,  we  would  have  to  start  by 
understanding  the  tasks  that  they  have  to  perform,  the  capabilities  of  the  autonomous  systems, 
and  the  failure  modes.  This  analysis  then  defines  the  interactions  required  between  the  humans  and 
the  robots  either  to  control  a  system’s  capabilities  or  to  react  to  its  deficiencies.  The  required 
interactions  are  then  mapped  to  the  interfaces  that  allow  the  humans  to  perform  the  proper 
interactions. 

In  this  context,  there  are  three  general  forms  of  interaction  modes,  which  parallel  autonomy 
levels:  remote  control,  teleoperation,  and  telerobotics  of  the  soldier/robot  interface. 

Remote  control  interactions  are  usually  accomplished  with  simple  operator  interface 
designs.  Operators  have  line  of  sight  to  the  unmanned  vehicle,  so  their  eyes  provide  the  sensory 
feedback  needed  for  vehicle  navigation  and  locomotion  control.  Feedback  to  control  specific  on¬ 
board  payloads  also  comes  from  direct  sensory  feedback  or  through  simple  displays.  Control 
interfaces  for  these  vehicles  usually  are  simple  in  design,  given  the  constrained  nature  of  the 
interaction.  Examples  of  such  vehicles  are  remote-controlled  model  airplanes  and  cars.  Military 
ground  examples  are  the  Mini-Flail  and  Panther.  These  types  of  systems  usually  require  one 
operator  per  vehicle.  The  components  of  these  systems  are  much  the  same  as  those  found  in  any 
vehicle-based  system.  What  is  unique  to  the  world  of  unmanned  vehicles  is  the  dominance  of 
electronic  information  flow  between  the  soldier  and  the  unmanned  vehicle.  There  is  the  vehicle 
that,  depending  upon  the  mission,  is  more  or  less  “smart.”  There  is  the  soldier- vehicle  interface 
consisting  of  display  and  controls.  Finally,  the  soldier  will  oversee  or  direct  the  system  to 
accomplish  the  mission  or  task  to  the  maximum  of  its  capability. 

We  cannot  overemphasize  the  importance  of  the  soldier  getting  required  information  in  a 
timely  manner  and  in  a  useful  form.  On  some  missions,  as  unmanned  vehicles  become  even  more 
capable,  soldiers  will  command  several  robots  and  robots  will  communicate  among  themselves  to 
solve  problems.  These  factors  require  the  development  of  interfaces  that  assure  timely  information 
flow  at  various  levels  of  employment  where  changes  occur  rapidly. 

Teleoperation  interactions  require  more  sophisticated  display  and  control  interfaces  because 
navigation,  locomotion  control,  task  execution,  and  vehicle  status  monitoring  are  not  performed 
by  line  of  sight.  Therefore,  soldier  interface  displays  must  be  capable  of  providing  appropriate  and 
timely  information  feedback  that  is  formatted  in  ways  that  allow  operators  to  successfully  control 
unmanned  vehicles  and  their  mission  equipment  packages.  Controller  interface  designs  must 
permit  operators  to  send  commands  to  the  vehicle,  based  on  display  information  feedback,  that 
are  timely,  appropriate,  and  accurate  enough  to  accomplish  the  mission  successfully.  Some  believe 
that  unmanned  vehicles  of  this  type  evolved  from  manned  vehicles.  More  specifically,  these 
include  manned  vehicles  designed  to  achieve  near  daytime  operational  capability  24  hours  a  day  in 
nearly  all  weather  conditions,  and  those  designed  to  operate  in  a  “closed  hatch”  mode  during 
NBC  operations.  To  achieve  the  above  goals,  many  of  these  manned  vehicles  employ  advanced 
sensors  that  provide  visual  and  auditory  feedback  about  the  environment.  These  sensors  often 
operate  in  ranges  of  the  electromagnetic  spectrum  (12,  IR,  RF,  MMWR,  and  LADAR6)  that 


6  12:  image  intensification;  IR:  infrared;  RF:  radio  frequencies;  MMWR:  millimeter-wave  radar; 
LADAR:  laser  radar. 
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exceed  the  direct  sensing  capability  of  the  vehicle  operator’s  eyes  and  ears.  Therefore, 
operational  capability  additional  to  that  of  the  unaided  human  sensory  system  can  be  achieved. 

The  visual  displays  used  in  conjunction  with  these  sensors  include  helmet-or  panel-mounted 
displays.  Operator  control  interfaces  to  slew  sensors  help  to  overcome  sensor  field-of-view 
limitations.  Helmet-mounted  displays,  the  sensors  are  often  slaved  to  the  movement  of  the  helmet 
to  provide  slewing  control.  Vehicles  having  the  above  sensor  capabilities  often  integrate 
locomotion,  navigation,  target  acquisition,  fire  control  and  vehicle  status  feedback  information 
into  a  limited  number  of  operator  interface  displays.  In  many  cases,  soldier  interfaces  designed  to 
control  these  vehicles  and  perform  mission  tasks  are  also  integrated  into  a  common  module. 

Many  of  the  control  system  outputs  are  electrical  or  electro-optical  signals  that  go  to  power- 
assisted  actuators.  Therefore,  a  transition  from  a  manned  to  unmanned  vehicle  fundamentally 
involves  providing  links  for  sending  signals  from  sensors  on  board  the  vehicle  to  soldier  display 
interfaces  in  remote  operator  stations.  Links  also  must  be  provided  to  send  signals  generated  by 
the  operator  control  interface  back  to  vehicle  actuators,  thus  enabling  soldiers  to  control  the 
vehicle  while  they  remain  out  of  harm’s  way.  It  is  no  surprise  that  remote-control  stations  often 
look  like  in-vehicle  control  stations.  The  assumption  is  that  whether  a  soldier  is  in  the  vehicle  or 
remote  to  the  vehicle,  the  display/control  interfaces  and  layout  should  be  similar.  Military  ground 
examples  of  this  would  include  the  Man-Portable  Remote  System  (MPRS). 

The  teleoperation  interaction  approach  usually  requires  one  operator  per  system.  Operator 
data  requirements  for  effective  control  and  display  usually  demand  large  bandwidth  for 
transmission.  Additionally,  locomotion  and  navigation  control  of  these  systems,  especially  ground 
systems,  is  difficult  because  of  sensor  and/or  display  limitations  and  lack  of  operator  motion  cues. 

The  bulk  of  the  current  requirements  for  unmanned  ground  systems  in  the  Army  are  in 
applications  of  remote-controlled  or  teleoperated  systems.  The  Joint  Program  Office  (JPO)  at 
Huntsville  is  focused  on  satisfying  these  requirements. 

Telemanagement  (aka  telerobotics)  interaction  approaches  require  a  teleoperation 
capability,  but  provide  more  capability  for  autonomous  action.  The  goal  is  to  provide  as  much  as 
possible  of  the  autonomous  capability  at  the  vehicle  level  in  order  to  reduce  the  data  transmission 
requirements  between  vehicles  and  remote  display  and  control  interface  sites.  Reducing 
transmission  requirements  to  reduce  bandwidth  requirements  is  critical.  In  this  design  approach, 
adding  more  autonomy  or  off-loading  actions  reduces  operator  workload  and  provides  increased 
performance  over  teleoperation.  It  can  also  enable  the  operator  to  operate  more  than  one  vehicle, 
i.e.,  one  on  many  (force  multipliers).  When  these  systems  become  smart  enough,  operators  will 
assume  higher-level  roles  and  interact  only  on  functions  and  tasks  that  a  priori  are  determined  to 
require  soldier  input.  Examples  might  include  planning  a  mission  that  would  entail  defining  its 
destination,  identifying  targets  to  kill,  and  intervening  when  emergencies  occur.  For  operational 
effectiveness,  however,  highly  autonomous  robots  may  require  operator  control  and  display 
interfaces  more  complex  than  those  required  by  teleoperation.  This  statement  is  based  on  the 
assumption  that  successful  intervention  in  emergency  situations  may  require  telepresence  or 
perceptual  immersion.  In  addition,  the  assurance  of  timely  and  appropriate  inputs  may  require 
sophisticated  decision-aiding  displays  and  their  associated  software.  A  current  ground  vehicle 
with  a  rudimentary  version  of  what  we  call  telemanagement  properties  is  the  ARL  XUV  vehicle 
that  uses  the  Real-Time  Control  System  (RCS)  architecture.  RCS  is  one  of  the  most  promising 
architectures  for  providing  “smart”  vehicle  control  systems.  (James  Albus  and  Alexander  Meystel 
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described  RCS  in  Engineering  of  Mind,  [2001 ]).  This  hierarchical  planning  and  control 
architecture  is  compelling  from  a  number  of  perspectives.  For  example,  it  is  isomorphic  with 
general  stereotypes  regarding  intelligent  behavior  generation,  management  and  organization. 

These  features  have  significant  implications  for  soldier-machine  interface  design.  The  RCS  version 
currently  employed  in  the  XUV  vehicle  is  capable  of  navigation  and  locomotion  control.  This 
capability  is  advanced  enough  that  it  elicits  anthropomorphic  comments  from  soldiers.  Example 
statements  include  “Watch,  he  (referring  to  the  vehicle)  is  going  to  make  a  turn,”  or  “He  is  trying 
to  make  up  his  mind.”  The  development  and  fielding  of  “smart”  and  “smarter”  unmanned  vehicles 
becomes  more  feasible  each  year  as  the  weight,  power,  cost,  and  size  of  computing  power, 
displays,  and  sensors  decreases. 

Figure  9  shows  that  as  the  level  of  autonomy  for  any  task  increases,  the  nature  of  the 
interaction  between  the  human  and  the  unmanned  system  changes.  The  human  goes  from  being  a 
detailed  task- level  controller,  requiring  full-time  interaction,  to  more  of  a  supervisory  role;  a  role 
that  is  more  instructional,  more  involved  with  what  to  do  than  with  how  to  do  it.  Feedback  comes 
more  by  notification  and  query  than  by  continuous  monitoring,  unless  there  is  an  emergency  or 
highly  complex  or  risky  task. 

It  is  interesting  to  correlate  the  information  in  Figure  9  with  that  in  Figure  7.  In  Figure  9,  the 
area  between  the  teleoperation  functions  and  the  autonomous  functions  is  a  critical  region — one 
where  some  autonomy  is  possible  but  is  not  robust  enough  for  full-time  reliance.  Humans  must 
take  control  and  teleoperate  the  robots  whenever  things  do  not  go  according  to  expectations.  The 
interactions  that  enable  the  humans  to  understand  the  context  of  the  failure  and  the  potential 
remedies  are  significant  challenges.  Very  little  work  has  been  done  on  computational 
representations  of  such  spatio-temporal  context  and  on  the  mixed-initiative  nature  of  the 
interactions. 
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uman  Robot  Interactions  Depend  on  Autonomy 


•  Mixed  Teams  (Humans 
and  Robots) 

•  Full  Autonomy  (Interaction 
at  the  level  of  missions  and 
commander’s  intent) 

•  Telemanagement 
(Infrequent  interaction, 
usually  exception -based) 

•  Teleoperation  (Interaction 
at  a  distance,  sensor  and 
communications  mediated) 

•  Remote  control  (direct 
visibility  of  the  robot) 
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Figure  10.  Interactions  and  Autonomy. 


For  force  multiplication,  it  is  desirable  that  individual  humans  control  multiple  robots.  Today, 
Predators  require  a  crew  of  24  humans  to  manage  all  aspects  of  a  mission.  In  future,  the  hope  is 
that  with  appropriate  autonomy  and  interaction  modalities  a  single  soldier  would  control  multiple, 
perhaps  four,  robots.  If  this  is  achievable,  the  value  to  the  Army  should  be  significant. 

The  question  “how  many  robots  can  one  person  control”  is  often  asked,  but  is  extremely 
challenging  to  answer.  The  answer  depends  upon  all  of  the  factors  we  have  discussed  in  the 
previous  sections:  the  level  of  autonomy  assumed,  the  level  of  intelligence  in  the  robot  (ability  to 
deal  with  unexpected  situations),  the  complexity  of  the  environment,  and  the  criticality  of  the  task. 
At  the  lowest  level  of  autonomy  (remote  control),  the  connection  between  a  controller  and  a 
robot  is  only  1 : 1 .  As  autonomy  increases,  a  soldier  might  be  able  to  task  a  robot  to  initiate  a 
mission  and  then  devote  attention  to  other  robots,  returning  periodically  to  check  up  on  the 
progress  of  individual  robot  mission.  In  an  environment  where  everything  works  as  planned,  there 
is  no  theoretical  limit  to  the  number  of  robots  a  soldier  could  control.  Air  traffic  controllers  in  the 
U.  S.  airspace  routinely  manage  the  positions  and  movement  of  dozens  of  aircraft  (primarily 
through  effectively  structuring  what  is  otherwise  an  unstructured  environment).  However,  in 
environments  that  one  cannot  structure  and  where  things  do  go  wrong,  the  commander  or 
operator  has  to  keep  a  mental  map  of  what  is  happening — which  limits  the  span  of  control.  For 
example,  in  a  standard  military  environment,  the  commander’s  span  of  control  is  usually  3  to  5 
subordinates.  If  we  assume  the  existence  of  robots  with  the  same  level  of  capability  as  humans, 
the  span  of  control  is  unlikely  to  be  much  greater  than  that.  This  span  of  control  is  similar  to  what 
one  finds  in  typical  civilian  organizations. 

In  the  future,  humans  will  have  to  work  with  multiple  robots  and  the  robots  will  have  to 
work  with  each  other.  Therefore,  some  argue  that  the  design  of  the  human  control  and  display 
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interface  with  “really  smart”  unmanned  machines  should  eventually  duplicate  the  same  modalities 
as  human-human  interactions,  on  the  assumption  that  one  simply  tells  the  machine  what  to  do  and 
when  to  do  it.  The  machine  then  figures  out  how  to  comply  and  verbally  reports  back  when  it 
completes  the  assigned  task  or  mission.  It  also  provides  interim  progress  reports  if  they  are 
desired.  Only  if  an  autonomous  unmanned  vehicle(s)  encounters  a  situation  beyond  its  capacity 
will  it  ask  for  external  human  or  machine  assistance.  This  capability  to  interact  and  dialog  in  a 
human-like  modality  is  particularly  important  when  multiple  soldiers  and  robots  work  together  in 
integrated  teams.  Each  needs  to  understand  intuitively  what  all  of  the  others  are  doing. 


4.1 


HUMAN-ROBOT  INTERFACES  AND  TODAY  S  ROBOTIC  SYSTEMS 


emote  Controlled  Robots 


APPLIQUE  -  Robotics  applied  to  existing  systems  DEDICATED  -  Built  specifically  as  a  robotic  system 


Figure  11.  Remote-Controlled  Robots 

Figure  1 1  graphically  shows  the  remote-controlled  ground  robots  being  developed  by  the 
Army.  Of  these,  only  the  Panther  and  the  Mini-Flail  are  operational  today. 
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RI  for  Remote  Controlled  Robots 
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Figure  12.  Interfaces  for  Remote-Controlled  Robots 

Figure  12  indicates  the  fairly  simple  level  of  the  interfaces  required  to  operate  such  remote 
controlled  robots.  The  key  challenges  in  such  displays  are  engineering  in  nature:  building  the 
displays  cost  effectively  and  making  them  rugged  and  survivable. 
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iTelerobotic/Autonomous  Human  Robot 
Interfaces 
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Figure  13.  Human-Robot  Interfaces  for  Intelligent  Semiautonomous  Robots 

Figure  13  shows  the  interfaces  that  have  been  developed  on  Army  programs  dealing  with 
intelligent  semiautonomous  robots.  The  photograph  on  the  top  left  shows  the  Demo  III  flat-panel 
display  in  use.  Demo  III  robots  are  tasked  by  using  a  point-and-click  interface  to  provide  the 
robots  with  waypoints,  destinations,  and  simple  constraints  (such  as  “hug  the  tree-line”). 

The  three  screen  captures  in  the  bottom  half  of  the  figure  show  the  visualization  interface 
under  development  for  the  Crew  Automation  Testbed  (CAT)  ATD  program.  This  interface  will  be 
used  to  task  the  Robotic  Follower  (RF).  The  interface  consists  of  three  panels  with  a  full  suite  of 
software  reprogrammable  buttons,  as  well  as  a  hardware  “steering-wheel”  interface  with  hardware 
buttons  and  controls. 

Not  shown  in  the  figure  is  the  robotic  interface  under  development  at  Fort  Knox  for  use  with 
the  robotic  simulators  in  the  Battle  Lab. 

The  panel  observed  that  even  though  there  was  overlap  between  the  work  of  contractors  on 
the  Demo  III  and  the  CAT/RF  programs,  the  user  interfaces  were  different.  There  had  been  no 
experimentation  on  Demo  III  to  identify  the  effectiveness  of  the  interface,  and  therefore,  no 
lessons  learned  that  could  be  transferred  to  the  CAT/RF  program.  Further,  the  Demo  III  interface 
seemed  to  be  primarily  a  mechanism  to  task  the  robot  and  to  allow  the  demonstration  of  the 
autonomous  capability  of  the  XUV  vehicle.  Similarly,  the  CAT  interface  was  an  exercise  in 
building  a  highly  reprogrammable,  flexible  crew  workstation,  rather  than  a  robot  controller  per  se. 

A  key  panel  finding  was  that  the  stovepiped  nature  of  development  within  the  S&T 
community  and  the  lack  of  user  pull,  allowed  the  various  communities  to  focus  only  on  their  own 
narrow  technical  goals.  Even  the  demonstrations  that  each  community  was  planning  were 
uncoordinated  and  not  focused  on  a  common  user-defined  problem  (i.e.,  FCS  Block  I  scenarios). 
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Courtesy  NASA  JPL 


Figure  14.  Human-Robot  Interfaces  Developed  at  JPL 

This  problem  is  illustrated  in  Figure  14  by  three  human-robot  interfaces  developed  at  the 
NASA  Jet  Propulsion  Laboratory  (JPL).  The  interface  on  the  top  left  was  developed  by  JPL 
researchers  for  the  DARPA  Tactical  Mobile  Robots  (TMR)  program.  The  interface  on  the  bottom 
left  is  the  tool  used  to  control  Sojourner,  the  first  Mars  rover.  The  figure  on  the  top  right  shows 
the  interface  for  the  Wide  Area  Telerobotics  System  (WITS)  that  will  be  used  to  control  the  next- 
generation  Mars  rover  in  a  worldwide  Web-enabled  distributed  manner. 

Again,  no  metrics  were  used  to  evaluate  the  efficacy  of  each  of  the  interfaces.  It  appeared 
that  each  development  team  had  its  own  human-robot  interface  subteam  that  “invented”  its  own 
interface  tool  from  scratch.  Without  appropriate  metrics,  no  mechanism  can  evaluate  aspects  of 
the  interface  that  operate  well,  or  aspects  that  need  to  be  further  developed. 
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'Advanced  Multimodal  Interfaces 
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Figure  15.  Advanced  Multimodal  Interfaces 


Figure  15  shows  two  research  interfaces  more  sophisticated  than  simple  mouse-windows 
mechanisms.  The  left-hand  figure  shows  a  multimodal  interface  developed  by  researchers  at  SRI 
International  on  a  recent  DARPA  program.  The  interface  uses  speech,  gestures,  and  contextual 
references  to  objects  in  a  scene  (as  seen  by  the  on-board  robot  camera)  to  command  the  robots. 
The  figure  on  the  right  shows  an  eye-tracker  developed  at  Sandia  National  Laboratories  that 
allows  the  wearer  to  refer  to  robot  commands  by  simply  glancing  at  appropriate  commands 
projected  into  a  heads-up  display. 

Since  neither  of  these  programs  is  tied  to  any  real  application,  it  is  impossible  to  ascertain 
whether  such  interaction  modalities  are  useful,  or  even  necessary.  While  they  represent  significant 
advances  in  terms  of  human-robot  “dialog,”  value  can  only  be  determined  within  the  context  of 
real  applications.  For  example,  will  ambient  noise  affect  speech  recognition  adversely?  What  will 
be  the  results  of  the  interaction  if  commands  have  to  be  constantly  repeated?  Such  questions  can 
be  answered  only  if  the  community  develops  a  core  set  of  metrics  and  performs  a  series  of 
experiments  in  realistic  environments. 

What  is  common  to  all  these  examples  is  that  they  focus  on  interfaces,  not  interactions;  they 
lack  a  systematic  evaluation  of  the  relationships  between  missions,  robot  capabilities  and 
interactions;  they  have  no  metrics  for  the  evaluation  of  performance;  and  they  constitute 
technology  demonstrations,  not  solutions. 

For  example,  the  Demo  III  vehicle  “resorts  to  full  teleoperation”  if  it  cannot  reason  about  the 
path  on  its  own  in  three  attempts.  So,  while  there  are  interaction  modes  in  the  interface  to  provide 
the  robot  with  waypoints  and  goals,  there  is  no  effective  teleoperation  mode.  Users  have  to  resort 
to  remote  control,  direct  observation  of  the  robot  from  a  chase  vehicle,  to  perform  the  error 
recovery  tasks. 
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The  research  community  has  recognized  this  problem.  A  recent  joint  D  ARP  A/National 
Science  Foundation  (NSF)  workshop  on  human-robot  interactions  concluded  that 
multidisciplinary  research  was  needed  in  the  area.  They  recognized,  as  we  did,  that  suitable 
metrics,  and  focused  experimentation  in  defined  environments  were  critical  to  the  success  of  the 
field.  For  further  information  on  the  conclusions  of  the  workshop,  please  refer  to: 
http://www.aic.nrl.navy.mil/hri/nsfdarpa/ 

4.2  INTERACTIONS  IN  OTHER  COMPLEX  SYSTEMS 

Systematic  experimental  work  should  be  conducted  to  identify  what  Army  functions  and 
tasks  can  benefit  most  from  the  use  of  unmanned  vehicles;  determine  which  class  of  interaction 
should  be  used  for  the  identified  tasks  and  functions;  and  specify  the  design  of  the  soldier  control 
and  display  for  the  selected  classes  and  functions/tasks.  This  effort  should  embrace  the  full 
repertoire  of  developmental  tools  including  analytical  efforts,  simulation,  field  experiments,  and 
prototype  development.  The  failure  to  perform  systematic  experimental  work  to  gain  effective 
design  guidance  for  interfacing  humans  with  advanced  semi- autonomous  systems  will  lead  to 
undesirable  outcomes.  Researchers  in  a  number  of  fields  learned  this  lesson  the  hard  way.  For 
example,  commercial  airlines  have  achieved  enhanced  system  performance  by  using  advanced 
flight  management  systems  that  are  electronically  or  electro-optically  linked  to  actuators  that 
control  aerodynamic  surfaces  and  engine  performance.  Such  systems  make  an  airplane  “smarter,” 
in  that  they  enable  the  airplane  to  perform  many  tasks  that  previously  required  the  full  time 
attention  of  the  crew.  The  introduction  of  these  technologies,  when  correctly  designed  and 
interfaced  with  the  crew,  has  not  only  led  to  safer,  more  efficient  flight,  also  has  done  so  at  a 
reduced  cost.  On  the  other  hand,  when  the  crew  interface  is  incorrectly  designed  catastrophic 
outcomes  can  occur.  For  example,  a  “smart”  airplane  may  take  some  “not  so  smart”  actions 
under  certain  conditions  that  are  unknown  by  the  crew.  When  the  crew  finally  becomes  aware  of 
the  problem,  appropriate  corrective  actions  may  be  too  late.  In  some  cases,  crew  interfaces  are 
designed  so  as  to  induce  input  instruction  errors.  In  the  worst  case,  these  errors  lead  to 
catastrophic  results  and  in  the  best  cases;  they  result  in  losses  of  time  and  fuel.  The  medical 
equipment  field  has  also  experienced  its  share  of  catastrophic  results  due  to  inadequate  control 
and  display  interface  design.  For  example,  during  cancer  therapy,  remotely  controlled  “smart” 
machines  administered  radiation  overdoses  to  patients  because  of  faulty  human  control  and 
interface  design.  Some  of  these  overdoses  killed  patients.  Three  Mile  Island,  and  many  other 
complex  human-machine  environments  have  resulted  in  adverse  consequences.  The  message  is 
clear:  attention  must  be  paid  to  the  human  control/display  interfaces  and  the  underlying 
architectures  of  complex  smart  systems  if  humans  are  to  successfully  interact  with  them  and  they 
are  to  achieve  their  design  objectives. 

During  this  study  we  tried  to  determine  two  things:  first,  whether  the  Army  has  work  in 
progress  to  identify  the  tasks  and  functions  that  would  benefit  most  from  the  application  of 
manned  vehicles;  and  second,  whether  work  to  develop  effective  soldier  interfaces  with  unmanned 
vehicles  was  also  in  progress.  Emerging  results  seem  to  indicate  that  for  the  most  part  this  was 
not  the  case.  Inarguably,  much  good  work  is  in  progress  in  a  number  of  places.  Unfortunately, 
we  found  very  little  quantifiable  data  from  systematic  efforts  to  identify  the  tasks  and  functions 
that  could  most  benefit  from  the  use  of  unmanned  vehicles  with  various  capabilities.  Similarly,  we 
found  very  little  data  to  support  the  design  of  soldier  interfaces  with  unmanned  vehicles  designed 
to  execute  military  tasks.  To  some  extent,  these  findings  are  understandable,  given  the  maturity  of 
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some  levels  of  unmanned-vehicle-enabling  technologies.  However,  history  shows  that  success 
depends  on  the  early  introduction  of  the  ultimate  operator  and  maintainer  (the  human)  into  the 
design  loop.  Designs  that  do  not  heed  human  control  interface  issues  produce  results  that  can 
have  catastrophic  consequences  or  at  the  very  least  will  degrade  system  performance. 

We  surveyed  a  number  of  automation-related  research  efforts  concerned  with  crew- station 
design  in  the  aviation  field.  The  record  shows  that  making  the  appropriate  task  assignments 
between  the  pilot  and  the  machine  as  well  as  correctly  designing  the  pilot-computer  interface  are 
essential  to  efficient  operations.  We  posit  that  the  same  is  true  of  the  soldier-robot  system 
interface. 

The  factors  that  drive  unmanned  system  interactions  are 

•  The  mission/operational  environment  and  overall  task  complexity 

•  The  desired  ratio  of  soldiers  to  unmanned  systems 

•  Line-of-sight  vs.  non-line-of-sight  operation 

•  Communication  bandwidth  availability 

•  Technology  availability  and  cost 

•  Force/team  OPTEMPO  and  degree  of  operation  risk  or  threat. 

These  factors  drive  systems  design  and  dictate  the  level  of  soldier-system  interaction  as  well 
as  the  technology  design  tradeoffs  that  in  turn  determine  interface  requirements.  The  interface, 
regardless  of  the  technology  and  degree  of  automation,  must  provide  a  soldier  with  the 
information  that  he  or  she  needs,  when  they  need  it,  and  in  a  format  that  allows  them  to  respond 
so  as  to  accomplish  the  mission.  Figure  16  summarizes  the  importance  of  the  interface  between  a 
human  user  and  a  robotic  system. 
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Why  Focus  on  HRI? 


•  It  is  important  to  the  success  of  the  long-term  FCS  vision 

Humans  and  automated  systems  will  have  to  be  integrated 
and  work  seamlessly  together  to  achieve  the  goals 

•  It  can  reduce  development  and  fielding  costs 

•  It  will  increase  the  speed  of  development 

•  It  can  reduce  human  interface  related  failures 


Literature  from  other  complex  systems 
shows  the  value  of  HRI 
Failure  of  HRI  can  be  costly 
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Figure  16.  Importance  of  Human-Robot  Interactions 


4.3  THE  VALUE  OF  HUMAN-ROBOT  INTERACTIONS 

We  can  only  evaluate  the  value  of  human-robot  interactions  by  examples  from  the  field  of 
Human  Engineering  (HE).  HE  looks  at  the  issues  of  integrating  humans  into  complex  systems  in 
an  effective  manner. 

There  are  two  major  ways  to  illustrate  the  value  of  sound  HE  efforts.  One  is  to  show  the 
negative  results  from  lack  of  appropriate  HE  involvement,  and  the  other  is  to  demonstrate 
positive  results  of  HE  activities.  The  following  sections  examine  the  value  of  the  HE  effort  from 
both  perspectives. 

4.3.1  Historical  Problems  from  a  Lack  of  HE  Efforts 

Lack  of  appropriate  HE  involvement  in  design  can  result  in  system  shortcomings  that  require 
costly  redesign,  produce  substandard  system  performance,  or  in  extreme  cases,  precipitate  system 
failures  endangering  life  and  equipment.  Many  problems  found  during  testing  and  evaluation  are 
evidence  of  the  lack  of  a  good  HE  effort  during  the  design  and  development  phase.  While  some  of 
the  problems  are  resolvable,  it  typically  costs  more  to  make  these  changes  in  the  later  stage  of  a 
program  development..  Problems  found  during  the  operational  phase  are  even  more  costly  to 
resolve.  Sometimes  such  problems  are  identified  only  after  critical  incidents.  Two  examples  are 
provided  to  illustrate  the  problems  that  can  occur  when  insufficient  attention  is  paid  to  the  HE 
aspects  of  design.  The  first  set  consists  of  several  well-known  disasters  that,  though  they  have 
multiple  causes,  resulted  at  least  partly  from  lack  of  adequate  HE.  The  second  set  is  a  sampling  of 
specific  lessons  learned  in  a  variety  of  HE-related  areas.  These  examples  are  provided  in  the  hope 
that  future  system  designs  will  benefit  from  previous  system  design  failures. 
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One  of  the  technology  areas  in  which  systematic  HE  work  is  beneficial  is  the  effective 
integration  of  humans  with  automated  systems  in  flight-deck  automation  and  glass-cockpit  design. 
In  this  area  a  community  of  scientists,  engineers,  operators,  and  regulators  works  closely  together 
to  study,  experimentally  evaluate,  and  iteratively  improve  the  interactions  between  pilots  and  the 
automated  technology.  Research  shows  that  in  general,  improvements  in  automation  significantly 
reduce  pilot  workload  and  stress.  On  the  other  hand,  improper  interactions  and  poor  interface 
designs  can  lead  to  failures:  “Since  these  aircraft  [glass  cockpits]  were  introduced  in  the  early 
1980s,  hundreds  of  incidents  and  a  few  fatal  accidents  have  occurred  in  which  pilot-computer 
interface  was  a  factor.”  [AWST  Jan  1995]. 

Some  of  the  “few”  accidents  in  which  automation  was  a  possible  contributing  factor  are 
listed  below. 

•  Boeing  707,  Jamaica,  New  York,  1962 

•  Lockheed  L101 1,  Miami,  Florida,  1972 

•  Douglas  DC10-30,  Luxembourg,  1979 

•  Douglas  DC- 10-30,  Boston,  Massachusetts,  1982 

•  Douglas  DC- 10,  New  York,  New  York,  1984 

•  Boeing  747-SP  near  San  Francisco,  California,  1985 

•  Airbus  320,  Mulhouse-Habsheim,  France,  1988 

•  Airbus  320,  Bangalore,  India,  1990 

•  Boeing  B767-300,  Suphan  Buri  Province,  Thailand,  1991 

•  Airbus  320,  near  Strasbourg,  France,  1992 

•  Douglas  MD11,  Alaska,  1993 

•  Airbus  A320  Warsaw,  Poland,  1993. 

4.3.2  Catastrophic  Accidents 

The  failure  to  adequately  consider  human  capabilities  and  limitations  in  system  design  can 
sometimes  have  disastrous  consequences,  as  illustrated  by  the  following  three  well-known 
incidents. 

Downing  of  Korean  Air  Lines  Flight  007 

Soviet  air-to-air  missiles  shot  down  KAL-007  on  1  September  1983,  when  it  strayed  into 
Soviet  air  space.  A  navigational  error  led  the  aircraft  approximately  365  miles  off  course,  placing 
it  over  Soviet  military  installations  at  Sakhalin  Island.  All  269  people  on  board  perished  after  a 
90-second  descent  into  the  Pacific  Ocean.  The  most  likely  cause  of  the  navigational  error 
concerns  the  inertial  navigation  system  (INS)  installed  in  this  large  passenger  aircraft.  The  aircraft 
had  three  INS  systems:  one  primary  system  and  two  backups.  Each  INS  could  be  programmed 
separately,  or  a  “remote”  mode  could  be  chosen  in  which  the  crew  programmed  only  the  primary 
INS  and  the  information  was  then  automatically  passed  to  the  two  backup  units.  To  ensure  that 
the  proper  coordinates  have  been  placed  in  the  system,  the  INS  checks  the  primary  INS 
coordinates  against  the  coordinates  entered  into  the  two  backup  units.  It  is  hypothesized  that  the 
crew,  to  save  time  and  energy,  chose  the  “remote”  mode  when  programming  the  INS  units  and 
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incorrectly  entered  the  flight  path  coordinates.  This  error  would  not  have  been  detected  when  in 
this  mode,  because  a  copy  of  the  incorrect  coordinates  would  have  been  used  to  check  the  original 
incorrect  coordinates.  This  INS  was  designed  to  reduce  workload  and  stress  to  the  aircrew. 
Unfortunately,  the  system  was  so  automated  that  it  caused  inactivity,  boredom,  and  complacency. 
Due  to  the  defective  interface  of  its  INS,  KAL-007  found  itself  off  course  and  in  unfriendly 
airspace,  which  led  to  tragedy  [Stein,  1983;  Malone,  1990;  Time,  1992]. 

Three  Mile  Island  Incident. 

On  28  March  1979,  operators  at  Three  Mile  Island,  a  nuclear  power  plant  in  Pennsylvania, 
made  a  series  of  mistakes  that  led  to  a  near  meltdown  of  the  plant's  reactor  core.  A  series  of 
equipment  failures  also  contributed  to  this  accident.  The  result  was  a  release  of  approximately 
1200  millirems  per  hour  of  radiation  into  the  environment,  forcing  the  evacuation  of  several 
thousand  residents  of  the  surrounding  area.  Fortunately,  there  were  no  deaths  as  a  direct  result  of 
the  incident.  The  near  meltdown  of  the  reactor  occurred  when  a  pilot-operated  relief  valve  at  the 
top  of  the  pressurizer  failed  to  close,  resulting  in  the  loss  of  a  pressurizer  steam  bubble  and  the 
loss  of  reactor  control  system  pressure  and  quantity.  The  indicator  light  on  which  the  operators 
relied  to  determine  the  position  of  the  relief  valve  led  them  to  believe  that  the  valve  was  closed, 
but  the  light  was  not  displaying  the  actual  system  state — rather,  it  showed  the  presence  of  a  signal 
commanding  the  valve  to  close.  In  other  words,  the  operators  believed  the  relief  valve  was  closed 
when  in  reality  the  valve  was  open,  despite  the  command  to  close.  This  belief  led  the  system 
operators  to  conclude  falsely  that  a  leak  had  occurred,  and  they  began  to  act  accordingly. 
However,  they  continued  to  make  errors  that  increased  the  volatility  of  the  system,  such  as 
confusing  reactor  B  with  reactor  A  (a  problem  directly  attributable  to  the  control  panel  layout). 
Two  hours  later  an  operator  who  had  recently  arrived  realized  that  the  relief  valve  was  at  fault 
and  initiated  proper  actions  to  correct  the  problem.  In  the  end,  an  investigation  by  the  Nuclear 
Regulatory  Commission  into  the  human  factors  aspects  of  the  accident  determined  that  “the 
human  errors  which  occurred  during  the  incident  were  not  due  to  operator  deficiencies  but  rather 
to  inadequacies  in  equipment  design,  information  presentation,  emergency  procedures,  and 
training”  [Malone  1990]. 

Crash  of  a  Passenger  Airliner  into  the  Florida  Everglades. 

In  1972,  a  Lockheed  L-101 1  descended  at  night  into  the  swamp  of  the  Florida  Everglades, 
killing  all  99  passengers  and  crewmembers  on  board.  The  ensuing  investigation  by  the  National 
Transportation  Safety  Board  (NTSB)  revealed  that  one  small,  burned-out  light  bulb  in  the  landing 
gear  indicator  set  in  motion  a  sequence  of  events  that  ended  in  a  completely  avoidable  tragedy. 
More  precisely,  the  response  of  the  flight  deck  crew  to  the  inoperative  bulb  ultimately  hardened 
the  last  links  in  the  chain  of  errors  that  led  to  the  eventual  crash  of  the  aircraft.  While  in  flight, 
each  of  the  three  crew  members  (flight  engineer,  first  officer,  and  captain)  fixated  on  solving  the 
same  problem,  an  aberrant  “landing  gear  down  and  locked”  bulb,  while  neglecting  to  notice  that 
the  autopilot  had  become  disengaged.  Quietly,  while  all  crew  members  were  attending  to  the 
same  non  emergency  condition,  the  aircraft  descended,  under  neither  human  nor  automatic 
control,  until  it  finally  came  to  rest  in  the  swamp  below  [NTSB,  1973]. 

4.3.3  Lessons  Learned 

The  following  lessons  learned  summarize  in  condensed  form  the  experiences  of  users  and 
managers  of  systems  whose  designers  failed  in  some  way  to  adequately  consider  human 
capabilities  and  limitations.  It  is  often  difficult  to  obtain  detailed  data  directly  related  to  such 
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problems,  since  these  could  be  used  to  indicate  an  error  or  tarnish  the  image  of  a  contractor.  To 
avoid  legal  ramifications,  the  lessons  learned  are  stated  in  very  general  terms,  and  references  to 
specific  defense  systems  and  manufacturers  have  been  omitted.  In  some  cases,  human  engineering 
was  involved  late  in  the  design  process  and  the  problems  were  turned  into  success  stories.  One 
major  lesson  learned  is  that  costly  engineering  changes  could  have  been  prevented  if  HE  had  been 
involved  earlier  in  the  systems  acquisition  process.  We  present  three  examples. 

Landing  Gear  Visibility  at  Night 

Failure  to  design  a  helicopter  landing  gear  so  that  it  remains  visible  to  the  landing  signal  crew 
at  night  can  result  in  a  wheels-up  landing,  causing  damage  to  aircraft,  safety  hazards  to  aircrew 
and  ground  personnel,  and  operational  loss  of  a  valuable  fleet  asset.  [Department  of  the  Air  Force 
1996], 

Attitude  Directional  Indicator  (with  no  Velocity  Vector 

Heads-up  display  without  velocity  vector  indicators  do  not  provide  a  flight  path  marker, 
leading  to  possible  situational  awareness  problems  for  the  pilot  [Air  Force  HSI  Office  1995. 

Night-Vision  Goggles  and  Exterior  Lighting. 

A  failure  to  provide  fleet  aircraft  with  exterior  lighting  compatible  with  the  use  of  night- 
vision  goggles  for  night  missions  can  result  in  mid-air  collisions  with  aircraft  that  are  not 
operating  within  the  flight  formation  [Department  of  the  Air  Force  1996].  (See  appendix  D  for 
more  examples) 

4.3.4  Benefits  from  Human  Engineering 

As  with  most  worthwhile  efforts,  HE  requires  an  investment  of  money  and  time  to  gain 
eventual  savings,  increased  total  system  performance,  safety,  and  user  satisfaction.  Typically,  an 
investment  in  HE  is  relatively  small  compared  to  those  in  other  system  creation  activities,  while 
the  return  is  relatively  high  (see  Table  3). 


Table  3.  Example  Benefits  from  HE 


System  Type 

Investment 

($) 

Total  Savings 

($) 

Time 

(Years) 

Reconnaissance  &  Light 
Attack  Helicopter 

74.9M 

3.29B 

20 

Attack  Helicopter 

2.3M 

268.8M 

20 

Nuclear/Biological/Chemical 
Reconnaissance  Vehicle 

60K 

$2-4M 

1 

Source:  Booher  [1997] 


HE  efforts  strive  to  optimize  a  system  to  (1)  permit  operator  and  maintenance  personnel  to 
achieve  the  required  performance  levels;  (2)  minimize  manpower,  personnel  and  training 
requirements;  (3)  achieve  the  required  reliability  and  effectiveness  of  personnel-system 
combinations;  and  (4)  enhance  operational  safety  by  avoiding  human  error.  These  benefits  can  be 
seen  in  overall  system  and  HE  testing  and  evaluation  reports.  Success  stories  such  as  the 
following  help  illustrate  the  importance  and  value  added  of  HE  efforts. 
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Center  High-Mounted  Brake  Lights. 

In  1985,  after  extensive  HE  studies  showing  positive  results,  high-mounted  center  brake 
lights  became  standard  equipment  on  all  new  passenger  cars  in  the  U.S.  Vehicles  equipped  with 
the  new  brake  lights  are  involved  in  50%  fewer  rear-end  collisions,  and  when  an  accident  does 
occur,  the  costs  of  repairs  are  50%  less.  It  was  estimated  that  126,000  reported  crashes  would  be 
avoided  annually  from  1997  on,  resulting  in  savings  of  $910  million  a  year.  These  benefits 
stemmed  from  a  $5  million  investment  in  HE  [Hendrick,  1996], 

Redesign  of  Aircraft  Throttle  Module 

An  oversensitivity  problem  (an  unacceptably  large  output  in  response  to  a  small  input)  was 
discovered  in  the  use  of  the  throttles  of  a  large  transport  aircraft  during  aerial  refueling.  After 
engineers  unsuccessfully  redesigned  the  throttles  without  HE  input,  the  HE  group  was  asked  to 
develop  a  solution.  The  HE  practitioners  collected  and  analyzed  data  and  identified  critical 
component  failures.  They  worked  with  designers  to  modify  the  throttles  by  reducing  the  spring 
force,  smoothing  the  cam  action,  and  adding  helper  handles.  The  resdesign  was  greeted  with 
overwhelming  pilot  acceptance  [Air  Force  HSI  Office  1995], 

Modification  of  a  Manufacturing  Facility 

In  the  first  year,  following  an  HE  evaluation  and  modification  of  a  manufacturing  facility, 
worker's  compensation  losses  dropped  more  than  75%,  from  $400,000  to  $94,000.  The  changes 
that  resulted  from  this  HE  evaluation  saved  the  manufacturer  $1.48  million  in  the  period  1990- 
1994  [Hendrick  1996], 

Transceiver  Operator  Panel 

Sound  HE  involvement  often  goes  unnoticed  because  of  the  flawless  way  a  system  operates. 
A  transceiver  operator  panel  for  the  control  of  an  airborne  computerized  communications  sending 
and  receiving  processor  was  designed  according  to  HE  principles.  A  task  analysis  fleshed  out  six 
major  system  requirements,  and  fit  the  system  into  an  existing  slot  on  the  flight  deck.  The  design 
was  integrated  into  the  operator's  concept  of  operation  so  well  that  upon  first  questioning  during 
T&E,  the  field-test  engineers  could  not  recall  using  the  system  at  all  [Shafer  1976]. 

4.3.5  The  Pervasive  Impact  of  Windows  and  GUIs. 

The  previous  examples  in  this  section  have  focused  on  the  problems  that  occur  when  user 
interaction  design  is  not  done  in  a  rigorous  or  systematic  manner.  The  computer  industry  presents 
one  significant  example  of  the  positive  value  of  developing  interactions  and  interfaces  in  an 
organized  manner — the  development  of  graphical  user  interfaces  (GUIs),  leading  to  the  present 
day  Windows™  operating  system. 

In  the  early  sixties,  the  US  Air  Force  funded  a  series  of  research  contracts  for  the 
examination  of  human  interactions  with  time-shared  computer  systems  that  were  then  starting  to 
come  into  being.  One  of  these  contracts  was  to  a  visionary  computer  scientist,  Douglas  Englebart, 
who  was  then  with  the  Stanford  Research  Institute  in  Menlo  Park,  CA.  Doug  focused  his  energies 
on  understanding  how  humans  would  interact  with  computer  information,  and  in  doing  so, 
developed  concepts  such  as  multimedia,  hypertext,  windows,  icons,  and  the  computer  mouse.  In 
the  fall  of  1968,  Doug  demonstrated  the  first  “personal  computing”  platform  with  a  mouse 
controlling  a  networked  computer  system  with  hypertext  linking,  real-time  text  editing,  multiple 
windows,  and  teleconferencing. 
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These  concepts,  evolving  through  the  lineage  of  Xerox,  and  Apple,  became  widely  adopted 
as  the  preferred  mechanisms  for  interacting  with  computer  information.  Arguably,  the  windows- 
type  interfaces  map  readily  into  the  cognitive  models  that  underlie  common  office  actions  such  as 
filing  information,  searching,  and  indexing.  It  is  this  mapping  that  makes  windows-type  interfaces 
"intuitive".  Our  challenge  to  the  robot  human  interaction  community  can  be  summed  up  in  one 
short  sentence:  "What  is  the  analog  of  the  desktop  metaphor,  in  the  field  of  robotics?" 

We  feel  that  if  such  an  intuitive  model  could  be  devised,  it  would  provide  a  significant  boost 
to  the  usability,  and  hence  deployment  of  robots  in  the  Army. 
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5  PANEL  FINDINGS 


indings 


•  No  focused  effort  on  the  human-robot  interaction  issues  anywhere 

Ad-hoc  interfaces,  built  by  engineers,  for  engineers 
Even  NASA  has  multiple  competing  approaches 
(programmatics  driven) 

•  No  push  to  get  autonomous  robots  into  actual  use 

No  clear  roadmap  for  FCS  Block  II  and  beyond 
Sandbox  mentality 

No  cross-branch  proponent  or  technology  advocate 
No  end-to-end  simulation  capability  that  compensates 

•  Significant  progress  in  the  Demo  III  program  since  ASB  last 
observed  the  capabilities 

However,  full  A-B  mobility  at  speed  and  across  all  terrain  is  not 
yet  possible 

NASA  (JPL),  Sandia,  and  others,  have  significant  programs 
underway  that  can  be  leveraged  for  Army  use 


Robotics  TF 
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Figure  17.  Findings 


5.1  SIGNIFICANT  ADVANCES  IN  FUNDAMENTAL  ROBOTICS  TECHNOLOGY 

Significant  progress  has  been  made  in  the  Demo  III  program  since  the  ASB  last  observed  the 
capabilities.  The  final  capabilities  demonstration  at  Fort  Indiantown  Gap  in  October  2001  showed 
a  convincing  capability  of  unmanned,  supervised  autonomous  systems  to  perform  a  meaningful  set 
of  RSTA  tasks.  Although  full  A-B  mobility  at  speed  and  across  all  terrain  is  not  yet  possible, 
ground  vehicle  robotics  technology  has  matured  sufficiently  that  useful  military  systems  can  be 
fielded.  Robotic  ground  vehicle  development  has  reached  a  point  where  user  input  is  needed  to 
carry  the  work  forward  to  fully  useful  robotic  combat  systems.  The  Real-time  Control  System 
(RCS)  is  one  of  the  best  control  system  architectures  to  achieve  greater  unmanned  vehicle 
autonomy.  The  Army  Vetronics  Technology  Integration  (VTI)  program  and  the  associated  Crew 
Automation  Testbed/Robotic  Follower  (CAT/RF)  Advanced  Technology  Demonstration  (ATD) 
program  are  starting  to  address  the  issue  of  integrating  Demo  III  mobility  and  perception 
capabilities  into  a  transitionable  system. 

5.2  NO  SYSTEMATIC  STUDY  OR  APPLICATIONS  OF  HUMAN-ROBOT 
INTERFACE  DESIGN 

There  is  no  systematic  application  of  basic  principles  to  the  matter  of  human-robotic 
interfaces.  All  robotic  systems  have  human  interfaces,  which  leads  to  an  assumption  in  the 
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research  community  that  the  problem  is  being  addressed.  Most  interfaces  that  we  observed  (Army 
programs,  NASA  programs,  etc.)  have  interfaces  designed  primarily  by  robotics  engineers  for 
their  own  use.  Even  interfaces  (such  as  those  on  the  NASA  space  programs)  that  have  been 
designed  by  committees  with  end-user  participants  have  not  evolved  according  to  a  systematic 
hands-on  analysis  of  the  actual  use  of  the  interfaces  in  real-world  conditions.  Other  technical  areas 
that  have  systematically  applied  HE  processes  in  the  past  (such  as  flight  deck  automation)  have 
reported  significant  measurable  value  from  the  application  of  systematic  techniques. 

There  is  a  lack  of  systematic  analytical  and/or  experimental  work  to  identify  the  combat 
functions  and  tasks  that  could  have  the  highest  payoff  from  the  use  of  unmanned  vehicles;  and  to 
define  the  soldier  interface  design  required  for  unmanned  vehicles  that  will  perform  combat 
functions  and  tasks.  This  is  especially  true  with  regard  to  telemanaged  systems 

Every  project  approaches  the  human-robotic  interface  problem  in  its  own  way.  The  usual 
programmatic  drivers  for  this  are  principal  investigators  who  are  focused  on  the  core  robotic  or 
platform  technologies  rather  than  interfaces  (for  example,  Demo  Ill’s  main  focus  is  on  developing 
perception  technology  and  not  on  the  robotic  system  per  se);  the  lack  of  a  systematic 
understanding  of  the  goals  of  the  system  and  the  tasks  expected  to  be  carried  out  by  the  elements 
of  the  system  (including  the  human  elements);  and  the  lack  of  hands-on  experience  with  the 
existing  robotic  systems  in  the  field  to  catalog  and  document  interaction  needs.  Acceptable 
metrics  for  human-robot  interfaces  are  also  lacking. 

5.3  DISCONNECT  BETWEEN  END  USERS  AND  DEVELOPMENT  PROCESSES 

Basic  robotics  technology  has  progressed  to  where  it  is  possible  to  incorporate  some 
elementary  ground  robotics  capabilities  into  FCS  Block  I  if  the  acquisition  system  were  structured 
to  move  technology  rapidly  into  the  field.  Robotic  ground  vehicles  still  do  not  have  a  “sponsor”  in 
the  way  that  aviation,  trucks,  guns,  missiles,  and  tanks  do.  To  be  successful,  FCS  program 
management  has  to  be  able  to  find  needed,  useable  military  or  commercial  technology  and  force 
its  rapid  maturation,  but  the  “normal”  processes  of  the  acquisition  system  cannot  deliver  the 
needed  technology  in  a  timely  fashion.  The  fielding  of  existing  developmental  robotic  ground 
vehicles  for  training  purposes  also  would  speed  up  combat  development  work  for  the  FCS,  but 
there  does  not  appear  to  be  any  clear  path  by  which  the  (robotic  vehicle)  work  being  done  within 
the  Army  R&D  community  can  find  its  way  into  the  FCS  development  program. 

The  Army  acquisition  system  appears  to  be  disconnected  at  every  transition  point  in  the 
developmental  process.  The  system  is  focused  on  formalism,  on  organizational  structure,  on 
funding  categories — and  on  a  zero  defects  philosophy  that  is  totally  intolerant  of  fielding  less- 
than-perfect  equipment.  An  output-oriented,  field-and-upgrade  approach  is  at  the  heart  of  the 
spiral  development  concept,  which  focuses  on  moving  new  technology  through  the  developmental 
process  and  into  the  field  as  quickly  as  it  can  be  made  useful.  A  recent  Defense  News  article 
points  out,  however,  that  Congress  does  not  want  to  buy  anything  until  it  has  been  completely 
tested  and  proven  [Kaufman  2002],  In  that  regard,  the  Army  Acquisition  System  is  in  step  with 
Congress,  which  may  be  very  unwilling  to  accept  the  notion  of  spiral  development.  The  Army 
often  is  accused  of  stovepiping  development  on  the  assumption  that  this  is  a  bad  way  to  manage 
R&D,  when  the  perceived  stovepipe  development  was  actually  quite  logical  and  served  very  well. 
Stovepiping  was  an  effective  way  to  develop  the  equipment  the  Army  needed  when  there  was 
relatively  little  technology  crossover  between  the  systems  concerned.  The  large  systems 
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developed  in  the  1970s  and  1980s — Ml  Abrams,  HMMWV,  HEMTT,  Apache,  Blackhawk, 
etc. — were  products  of  that  type  of  development,  as  were  their  many  predecessors  developed 
from  1940  tol970. 

Over  the  past  60  years,  however,  as  technology  became  more  complex  and  system  costs  rose 
astronomically,  local  control  over  the  development  process  has  continually  devolved  toward  a 
centralization  of  decision-making  in  the  Department  of  the  Army  (DA)  and  the  DoD.  At  the 
same,  time  much  formerly  in-house  Army  R&D  work  has  migrated  to  industry,  with  a  consequent 
degeneration  of  government  “engineering  muscle”  into  “bureaucratic  gristle.”  The  time  needed  to 
develop  and  field  a  new  major  item  has  at  least  doubled  since  1960.  There  still  are  many  pockets 
of  solid  technical  capability  in  the  Army  system,  some  of  which  we  encountered  in  the  course  of 
this  study.  But,  like  a  species  being  driven  toward  extinction  by  piecemeal  habitat  destruction, 
they  increasingly  are  isolated  from  the  interaction  with  major  system  developments  that  is 
essential  both  for  them  to  contribute  and  to  survive. 

The  FCS  development  program  forces  all  of  the  Army  Acquisition  System’s  faults  and 
problems  out  into  the  open.  It  demands  the  rapid  fielding  of  new  technology,  and,  being  a 
“system  of  systems”  it  chops  across  the  “stovepipes.”  The  Army’s  response  is  to  bypass  FCS 
development  by  turning  to  industry  for  a  “lead  system  integrator”  (LSI)  to  integrate  the  disparate 
technologies  involved  into  a  coherent  combat  system.  It  remains  to  be  seen  how  well  this 
approach  is  going  to  work.  The  gold  standard  for  such  a  development  program  is  the  Navy’s 
1950s  Polaris  program,  a  tough  standard  to  meet  with  regard  to  both  quick  response  and  useful 
output.  The  Army  materiel  acquisition  organization  is  highly  compartmentalized,  and  putting 
together  a  project  management  structure  that  can  quickly  mobilize  its  resources  for  the  FCS  will 
be  challenging  at  best.  Technologies  such  as  UGVs  may  get  lost  in  the  shuffle.  In  the  course  of 
its  deliberations  over  the  past  10  months,  the  Human- Robot  Interface  panel  saw  considerable 
reason  to  believe  that  is  a  likely  outcome  for  the  very  promising  robotics  work  currently  going  on 
in  the  Army  laboratories  and  Research,  Development  and  Engineering  Centers  (RDECs).  The 
FCS  program  management  structure  must  be  set  up  to  prevent  this  outcome  since  the  FCS  cannot 
become  a  successful  fighting  system  without  its  robotic  subsystems.  In  fairness,  however,  we  must 
point  out  here  that  it  is  easier  to  apply  spiral  development  and  still  exercise  adequate  configuration 
control  over  a  fleet  of  less  than  50  large,  very  expensive  capital  ships  [none  of  which  are  ever 
exactly  alike,  anyway!]  than  it  is  to  control  the  configuration  of  a  fleet  of  tens  of  thousands  of 
combat  and  tactical  vehicles.  Anyone  who  has  ever  dealt  with  serial  production  and  maintenance 
of  equipment  in  the  field  should  be  a  serious  believer  in  rigorous  configuration  control.  This  is 
one  of  the  fundamental  weak  points  of  the  spiral  development  approach.  In  the  Army,  spiral 
development  could  easily  lead  us  to  the  German  Werhmacht's  WWII  problem  of  ground  vehicle 
populations  that  were  riddled  with  more  or  less  undocumented  running  production  changes,  to 
mention  only  one  problem.  Modern  computer  systems  should  enable  us  to  cope  with  spiral 
development  today,  but  no  one  should  think  it  will  be  easy  or  foolproof. 

The  FCS  concept  of  a  system  of  manned  and  unmanned  vehicles,  ground  and  air,  is  not  new. 
Dr.  James  Albus  of  NIST7  proposed  the  basic  idea  at  the  start  of  the  1980s  in  an  address  to  an 
ADPA  Combat  Vehicles  Conference.  Dr.  Albus  referred  to  the  concept  as  a  system  of  “swarm 


7  NIST:  National  Institute  of  Standards  and  Technology;  ADPA:  American  Defense  Preparedness 
Association. 
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vehicles”:  the  unmanned  air  and  ground  vehicles  in  the  swarm  would  both  protect  the  manned 
control  vehicle  at  the  center  of  the  swarm  and  serve,  collectively,  as  its  primary  weapon.  He  was 
not  the  only  person  to  suggest  this  general  approach  to  future  combat  vehicles. 

W.  J.  Whelan  [1982]  laid  out  a  novel  concept  for  a  future  fighting  vehicle  system  of  small 
robotic  fighting  vehicles  under  the  direction  of  a  larger  manned  vehicle.  The  concept,  while  perhaps 
prophetic,  realistically  was  not  testable  at  the  time  it  was  proposed,  nor  was  it  possible  then  to 
predict  with  credibility  when  developments  in  computing  technology  might  make  it  possible  to 
build  such  a  system.  When  H.  H.  Dobbs  proposed  a  similar  approach  [1993]  the  technology 
picture  had  cleared  considerably,  and  the  explosive  growth  of  computational  power  over  the 
proceeding  decade  pointed  to  such  military  robotic  vehicle  systems  becoming  practical  in  the 
foreseeable  future.  The  article,  however,  received  little  comment.  At  that  time  the  Armor 
community  was  still  in  the  afterglow  of  its  brutally  decisive  success  in  Operation  Desert  Storm. 
Not  one  U.S.  tank  crewman  was  lost  to  enemy  action  during  the  100  hours  of  the  Desert  Storm 
land  battle,  while  literally  every  Iraqi  force  that  tried  to  fight  them  was  totally  destroyed  and 
suffered  enormous  loss  of  life.  The  Army  focus  then  was  on  a  “Super  Ml”  as  the  future  successor 
to  the  already  dominant  Ml  Abrams  series  of  Main  Battle  Tanks  (MBTs).  The  ASB  Tank 
Modernization  Study  of  1995-1998  reflected  that  focus.  Robotic  vehicles  were  discussed  only  in 
an  appendix.  However,  military  robotic  vehicle  research  did  continue  throughout  the  1980s  under 
the  direction  of  the  DoD  land  warfare  systems  specialists  and  DARPA.  Congress  created  the 
DoD  Joint  Robotics  Program  (JRP)  in  1989  to  consolidate  all  such  efforts.  The  Unmanned 
Ground  Vehicle/Systems  Joint  Project  Office  (UGV/S  JPO)  was  established  at  Redstone  Arsenal 
in  Huntsville,  Alabama  to  work  with  ground  robotic  vehicle  systems.  Members  of  this  panel  were 
briefed  on  some  of  their  work  during  our  visit  to  the  U.S.  Army  Aviation  and  Missile  Command 
(AMCOM).  The  Board  on  Army  Science  and  Technology  (BAST),  an  element  of  the  National 
Research  Council,  stated  in  its  1992  report  on  the  Army  of  the  21st  century  that  “the  core  weapon 
of  the  20th  century  has  been  the  tank.  The  core  weapon  of  the  21st  century  may  well  be  the 
unmanned  system  operating  mostly  under  computer  control  [i.e.,  autonomously]  with  human 
supervision.” 

The  Demo  III  work  that  some  study  group  members  observed  at  Fort  Indiantown  Gap  began 
in  1998  [Bomstein  and  Shoemaker  1998].  At  that  point  in  time  management  of  the  work  shifted 
from  the  DoD  and  DARPA  to  the  Army,  and  ARL  and  the  U.S.  Army  Tank  Automotive  and 
Armaments  Command  (TACOM)  became  involved.  The  involvement  of  TACOM,  which  is 
responsible  for  the  development  of  vehicle  and  armament  systems  for  the  Army  was  a  clear 
indication  that  robotic  ground  vehicles  were  approaching  the  point  of  practical  application  in  the 
field. 

By  the  end  of  the  1990s  it  also  had  become  possible  to  predict  with  reasonable  accuracy 
when  a  given  level  of  computational  capability  would  become  available  at  a  chosen  cost  level. 

The  Impact  Matrix  for  Computational  Technology  [Dobbs  2002]  shows  such  estimates  at 
nominally  current  PC  price  levels.  Professor  Moravec,  one  of  the  most  knowledgeable  people 
working  on  robotic  intelligence  and  sensing,  estimates  that  machines  will  reach  human-level  speed 
(108MIPS)  by  the  year  2040  [Moravec  1999].  Current  supercomputers  already  process  data  at 
about  4  ?  106  MIPS,  and  the  goal  of  the  IBM  Corporation  Blue  Gene  project  [DeCusatis  2001], 
first  announced  in  2000,  is  to  build  a  supercomputer  with  a  speed  of  over  1010  MIPS  by  2005,  one 
hundred  times  Professor  Moravec's  estimate  of  human-level  speed.  This  supercomputer  will  need 
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approximately  1013  bytes  of  memory.  The  price  of  supercomputer  speed  usually  falls  to  PC  levels 
by  about  20  years  after  its  introduction;  the  price  of  computer  memory  capability  has  always 
closely  followed  that  of  computational  speed.  Professor  Moravec’s  estimates  of  the  time  when 
robots  will  reach  human  mental  capability  thus  may  appear  conservative.  However,  note  that 
although  computational  speed  and  memory  capacity  are  necessary  conditions  for  intelligence, 
they  are  not  necessarily  sufficient  conditions!  No  one  yet  knows  what  the  sufficient  conditions 
are.  A  great  deal  of  work  also  remains  to  be  done  in  the  areas  of  sensors  and  the  interpretation  of 
their  outputs  before  fully  autonomous  machines  become  practical. 

Nevertheless,  the  U.S.  military  is  moving  strongly  toward  robotic  vehicle  systems  for  many 
missions,  including  combat,  in  the  near  to  mid  term.  Congressional  pressure  to  move  in  that 
direction  also  has  developed  [Wolfe  2000],  Army  leadership  now  appears  to  understand  the 
necessity  of  robotic  vehicle  systems,  a  capability  inherent  in  the  FCS  concept.  Whether  the  Army 
in  the  field  understands  yet  is  less  clear,  and  whether  the  acquisition  system  can  successfully 
develop  and  field  a  fightable  initial  version  of  the  FCS  in  the  appropriate  time  frame  is  open  to 
question.  Fielding  even  one  of  the  robotic  units  derived  from  the  Demo  III  work  with  the  Block  I 
version  of  the  FCS  would  both  improve  the  system’s  survivability  and  let  the  soldiers  begin  to 
learn  to  use  the  FCS  in  the  ways  that  will  make  it  a  successful  weapon  system.  The  HRI  panel 
believes  that  the  Army  can  accomplish  this  goal. 

For  the  present  (until  the  FCS  comes  on  the  scene),  the  panel  believes  that,  with  a  minimum 
of  changes,  the  Demo  III  robots  they  observed  in  November  at  Fort  Indiantown  Gap  are  suitable 
for  issue  to  troops  in  well-defined  environments  for  testing  and  training.  These  activities  would 
provide  essential  feedback  to  both  the  FCS  developers  and  the  combat  development  community 
and  would  ensure  that  the  FCS  was  far  more  capable  when  initially  fielded.  The  benefits  of  this 
approach  should  greatly  outweigh  the  moderate  cost  of  the  equipment  involved.  The  interim 
brigades  might  be  the  best  choice  for  the  initial  fielding  of  this  equipment. 
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6  PANEL  RECOMMENDATIONS 


ecommendations 


•  Ft.  Knox  and  TRADOC  should  specify  an  operational  architecture  for  the  use 

of  followers,  teleoperated,  and  autonomous  ground  robots 

Experiments  with  users  should  form  the  basis  of  the  operational  architecture 
Field  experience  with  existing  autonomous  robots  (such  as  Demo  III),  in 
experimental  environments  such  as  the  NTC,  and  with  the  National  Guard 
would  provide  a  rigorous  baseline  (ground  truth)  for  the  operational 
architecture 

•  PEO  FCS,  and  the  OFTF,  supported  by  the  FCS  LSI  should  formulate  a  Block  1 
human-robot  interaction  architecture  consistent  with  the  FCS  ORD  in  time  for 
the  FCS  Milestone  B  decision 

Robotic  followers  and  autonomous  RSTA  robots  are  technically  feasible  in 
the  FCS  SDD  timeframe 

•  ARL  should  immediately  create  and  lead  a  S&T  program  aimed  at  developing  a 

technical  architecture  for  human-robot  interactions  (focused  on  autonomous 
ground  robots) 

HRED’s  MANPRINT  activity  could  be  delegated  to  take  on  this  role 
Collaboration  with  DARPA  may  be  appropriate  to  develop  and  mature 
enabling  technologies  (Potential  synergies  with  IXO,  ATO,  and  IPTO) 
Collaboration  with  application-focused  RDECs  is  critical 
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Figure  18.  Recommendations 


6.1  SUMMARY  RECOMMENDATION 

The  panel  recommends  that  the  Army  move  rapidly  to  create  a  strong,  technology  focused 
program  for  understanding  and  developing  the  technologies  for  human  robot  interactions,  with 
particular  emphasis  on  autonomous  ground  robots.  We  recommend  that  such  a  program  be 
grounded  in  well-articulated  operational  needs,  and  be  oriented  towards  the  transformation  and 
objective  force  requirements.  The  proposed  efforts  should  augment  and  build  on  the  successful 
robotic  ground  vehicle  work  that  has  been  done  in  the  Army  laboratories  (particularly,  ARL)  and 
RDECs — most  specifically  the  Demo  III  and  related  ground  robotics  programs. 

To  this  end,  we  make  three  specific  recommendations. 


6.2  DETAILED  RECOMMENDATIONS 

1.  Ft.  Knox  and  TRADOC  should  specify  an  operational  architecture  for  the 

use  of  robots,  particularly  followers,  teleoperated  systems,  and  autonomous 
vehicles. 

•  To  be  useful,  the  operational  architecture  must  be  baselined  with  experimental 
data  gathered  in  operationally  relevant  scenarios,  and  in  realistic  operational 
environments. 
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•  To  this  end,  we  suggest  that  the  Army  apply  additional  resources  to  initiate 
accelerator  low  rate  initial  production  (LRIP)  fielding  of  the  Demo  III  RSTA 
robots  with  organized  feedback  to  the  supporting  R&D  programs,  and  constant 
upgrades  to  the  fielded  robots — total  deliberate  integration  on  a  real-time  basis 
of  (1)  the  material  development  process,  (2)  the  combat  development  process, 
(3)  troop  training,  and  (4)  lessons  learned  from  combat.  The  Human-Robot 
Interface  panel  is  not  aware  that  the  Army  has  done  this  before  except  on  a  very 
limited  basis  during  wartime.  In  this  environment  the  difference  in  the  nature  of 
robots  from  previous  equipment  should  surface. 

•  Research  should  continue  in  improving  the  core  robotic  technologies 
(particularly  perception,  navigation,  and  mobility).  The  most  important  aspect 
of  a  robotic  weapon  system,  its  (obviously  still  limited)  ability  to  “think”,  will 
improve  in  the  field,  due  to  its  “experiences”  in  dealing  with  its  environment 
(soldiers  and  missions)  and  to  relatively  low-cost  software  and  hardware 
upgrades  to  its  ability  to  learn. 


•  Focus  robotic  R&D  on  making  the  following  improvements  in  robot  capability 
and  self-sustainment  ability 

1 .  Increased  temporary  memory  (RAM)  and  processor  speed 
(a  factor  of  10?  is  available  now) 

2.  Permanent  memory  (must  be  rugged) 

3.  Sensory  capability  to  “see”  and  understand 

4.  World  model  software  (its  local  world:  terrain,  weather, 
mission  execution,  “its  soldiers,”  etc.) 

5.  Ability  to  manipulate  its  environment,  support  its  own 
needs 


6.  Ability  to  communicate  with  humans,  electronically  and 
verbally 

7.  Improved  automotive  capability 

2.  PEO  FCS,  and  the  OFTF,  supported  by  the  FCS  LSI  should  formulate  a 
Block  1  human-robot  interaction  architecture  consistent  with  the  FCS  ORD  in 
time  for  the  FCS  Milestone  B  decision 


•  Having  a  user  community  “pulling”  the  autonomous  robotics  technology  is  an 
invaluable  motivator  for  the  future  development  of  the  field.  As  we  have 
indicated  in  the  report,  autonomous  operation  is  a  key  need  for  FCS.  FCS  and 
the  Objective  Force  Task  Force  are  therefore,  the  most  appropriate 
“customers”  who  should  drive  for  technology  adoption,  and  define  an 
architecture  for  human-robot  interactions. 
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3.  ARL  should  immediately  create  and  lead  a  S&T  program  aimed  at  developing 

a  technical  architecture  for  human-robot  interactions  (focused  on 
autonomous  ground  robots) 

•  HRED’s  MANPRINT  activity  could  be  delegated  to  take  on  this  role.  ARL 
should  establish  an  R&D  program  to  systematically  review  the  possible  forms 
of  human-robotic  interfaces  and  the  work  that  has  been  done  in  this  area,  and 
develop  guidelines  for  the  development  of  future  military  robotic  systems.  The 
program  should  consist  of  an  analytical/experimental  effort  to: 

A.  Identify  the  combat  functions/tasks  that  could  benefit  most  from  the 
application  of  unmanned  vehicles 

B.  Determine  the  interaction  class  that  best  fits  the  identified  combat 
functions  and  tasks  that  have  the  highest  payoff  if  unmanned  vehicles 
are  used 

C.  Provide  design  guidance  for  the  unmanned  vehicle  and  the  operator 
control  and  display  interfaces  that  will  provide  the  desired 
operational  capability 

•  Collaboration  with  DARPA  may  be  appropriate  to  develop  and  mature  enabling 

technologies  (Potential  synergies  with  IXO,  ATO,  and  IPTO) 

•  Collaboration  with  application-focused  RDECs  is  critical. 
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ADI 

Attitude  Directional  Indicator 

ADPA 

American  Defense  Preparedness  Association 

AMCOM 

Aviation  and  Missile  Command 

ARL 

Army  Research  Laboratory 

ARL-XUV 

Army  Research  Laboratory — Experimental  Unmanned  Vehicle 

ASB 

Army  Science  Board 

ATD 

Advanced  Technology  Demonstration 

ATO 

Advanced  Technology  Office  (DARPA) 

BDA 

Battle  Damage  Assessment 

BAST 

Board  on  Army  Science  Technology  (National  Academies  of  Science, 
Department  of  Military  Science  and  Technology) 

C4ISR 

Command,  Control,  Communications,  and  Computers,  Intelligence, 
Surveillance  and  Reconnaissance 

CAT/RF 

Crew  Automation  Testbed/Robotics  Follower 

DA 

Department  of  the  Army 

DARPA 

Defense  Advanced  Project  Research  Agency 

DoD 

Department  of  Defense 

FCS 

Future  Combat  System 

GUI 

Graphical  User  Interface 

HE 

Human  Engineering 

HF 

Human  Factors 

HFE 

Human  Factors  Engineering 

HRI 

Human-Robot  Interface 

HSI 

Hyperspectral  Imaging 

INS 

Inertial  Navigation  System 

IPTO 

Information  Processing  Technology  Office  (DARPA) 

IR 

Infrared 

IXO 

Information  Exploitation  Office  (DARPA) 

JPL 

Jet  Propulsion  Laboratory 

JPO 

Joint  Program  Office 

JRP 

Joint  Robotics  Program 

KAL 

Korean  Air  Lines 

LADAR 

Laser  detection  and  ranging 
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LRIP  Low-rate  initial  production 

LSI  Lead  System  Integrator 

MAY  Manned  Aerial  Vehicle 

MBT  Main  Battle  Tank 

MIC  Methyl  isocyanate 

MIPS  Million  Instructions  per  Second 

MMWR  Millimeter  Wave  Radar 

MOUT  Military  Operations  in  Urban  Terrain 

MPRS  Man-Portable  Remote  System 

MPT  Military  potential  test 

NASA  National  Aeronautics  and  Space  Administration 

NBC  Nuclear,  Biological  and  Chemical 

NIST  National  Institute  of  Standards  and  Technology 

NRC  National  Research  Council 

NTC  National  Training  Center 

NTSB  National  Transportation  Safety  Board 

OFTF  Objective  Force  Task  Force 

OFW  Objective  Force  Warrior 

ONR  Office  of  Naval  Research 

OPFOR  Opposing  Forces 

OSD- ATL  Office  of  Secretary  of  Defense  -  Acquisition,  Technology  and  Logistics 

PC  Personal  Computer 

PEO  Program  Executive  Office 

PI  Principal  Investigator 

Psig  Pounds  per  square  inch  gauge 

R&D  Research  and  Development 

RAM  Random  access  memory 

RCS  Real-time  Control  Systems  (other  contexts.  Radar  Cross  Section) 

RDEC  Research,  Development,  and  Engineering  Center 

RSTA  Reconnaissance,  Surveillance,  and  Target  Acquisition 

S&T  Science  and  Technology 

T&E  Test  and  Evaluation 

TACOM  U.  S.  Army  Tank  Automotive  and  Armaments  Command 

TIC  Tactical  Information  Coordinator 

TMR  Tactical  Mobile  Robotics  (DARPA  Program) 
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TOR 

TRADOC 

UAV 

UCAY 

UGCV 

UGV/S 

UMV/S 

YTI 

WITS 


Terms  of  Reference 
Training  and  Doctrine  Command 
Unmanned  Aerial  Vehicle 
Unmanned  Combat  Air  Vehicle 
Unmanned  Ground  Combat  Vehicle 
Unmanned  Ground  Vehicle/System 
Unmanned  Vehicle/System 
Vetronics  Technology  Integration 
Wide-area  Telerobotics  Systems 
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ITe  Unmanned  Ground  %r chicle  Roadmap  wa*  developed  10  provide  *  sirucmre 
for  organizing  SAT  programs.  in  support  o  I  Ann  y  imascras  lor  a  period  of  more 
than  20  )*ais.  The  complexities  of  die  anticipated  jiussaoji*.  the  environments  m 
which  they  will  he  cam  eel  out,  and  ihe  nature  of  (he  UCft  leehnolopies 
ihen^dvc^havc  dictated  that  the  Roadmap addf  the  Army'-*  needs  at  two 
distinct  kw  els  of  detail  For  isear-lo  jii  miiaaoni  and  erasequent  J  needs.  the 
Roadmap  presides  ub^tjuin  jJ  derail  (&aa  Section  2  1 1  For  mid-  and  long-term 
missions  and  needa  1  sections  2.2  through  2.5  h  the  Roadmap  rraommendnlfom 
are  broader,  m  is  appropriate,  given  the  uncertainties  in  how  both  die  Army  V 
mia&kmi  for  L'tiVi  and  robotic  SA  T  will  evolve, 

1.14  A  Wold  about  Autonomy 

I  Ik  IKtS"  user  community  and  theUGV  RAD  o.-iiiiiiiimi>  cadi  - . uvnlyurw 

die  word  “autonomy”  m  discussions  of  gods,  m  in  “foday  wc  are  at  leleopcmhon 
and  sometime  in  the  future  well  he  al  autDaomy."  Although  the  word  Ls  useful  a* 
shorthand  iW  conveying  a  very  general  goal,  its  fandamenld  ambiguity  Creates 
objuaclcs,  both  for  those  who  would  clarify  what  capabilities  are  needed  and  for 
diuse  who  would  determine  what  research  Lo  puisne 

Why  does  ihe  comiminily  c.tre  uboiii  mobility  .iiiionomviu  or  otherwise? 
Consider  a.  commonplace  analogy  a  trip  to  the  store  lo  shop  for  groceries 
Mobility  is  die  meaii>  by  which  one  gets  to  the  grocery  srorejiowtvet  tlie  valuc- 
jdded  pan  of  the  trip  is  not  the  drive  to  die  slcre  but  what  happen^  in  die  store. 
Most  Airny  applications  envinioned  for  UQVs  are  similar  in  that  the  value-added 
part  of  a  mission  comes  at  ihe  end  of  the  journey  i  In  this,  VQVa  are  of  course  no 
differs i li  from  manned  Ibices.)  Thu--,  there  U  great  moiit  alion  to  develop  the 
capability  lo  can  y  out  the  nofi-\  jlne-added  portion  of  miKiioni;  with  high 
effectiveness. 

How  did  the  community  come  lo  the  conclusion  that  vehicles  need  autonomous 
mobility  ?  A  Ioj  Lg  hiMor y  of  experimentalio*  jdiows  that  remote  onlrof  of 
tdiiclv*  * ^pemti ^'•ri  with  no  visual  input  lo  the  remote  operat oi  i *  i n< ITecti ve, 
Ftinherruore,  even  ideaptmU^n-  when  visual  input  is  provided  i--.  not  effective, 
especially  when  die  UGVamust  keep  pace  with  u  maimed  force  moving  ciu^- 
couniry  There  are  not  jet  agreed-upon,  proven*  and  effective  meins  of 
impIcnieniiTig  the  non -value-added  portion  of  nwsi  important  iiiu.^iis:  geHmg 
die  UOV  to  i  point  where  ii  can  cany  out  its  mission  objective,  Yctp  without  ihe 
meajus  lo  perform  die  noil-value- added  pennon  of  the  mi&acm  effbcin  ely  and  with 
confidence,  the  Army  will  not  te  able  lo  fully  achieve  its  expectation*  for  UOV*, 

Why  is  autonomous  mobility  &v  hiard?  C  onsider  another  analogy:  the  re-entry 
*  elude  on  a  king-range  missile.  Tlie  U.S.  ha&  demonstrated  vehicles  dial  can 
seme  atmospheric  disturbance*  and  adjust  then  trajectory  to  erwure  high  end- 

point  accuracy.  Conventional  wisdom  holds  that  effective  1  Hi  Vs jit  have  a 

comparable  characteristic:  they  must  L low  w  here  they  are,  where  they  want  to  get 
lo,  and  how  to  adjust  iheii  uajociones  for  tme.%pecied  disturbances  along  line  way. 
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For  VirnsMU  ivaiotis,  including  marc  diflWiili  Hdi*tuxbEinc&  1  md  vastly  more 
important  cl  kcb  of  "clutter"  oil  llic  season,  Uus  is  j  much  more  difficult  Link  far 
UGVi  than  for  re-en«r  y  vehicles. 

Whai  can  be  done  about  iJik?  to  begin  with.  discimitini  of  autonomy  are  fraught 
with  difficulty  because  autonomy  has  not  been  defined  lh  engineemuj  lerms. 

Ai  though  the  UGV  community  generally  subscribes  to  die  idea  thai  there  are 
differing  levels  of  antono-my.  wilhonl  engineering,  dcfiniiiorii!  (honker  community 
mmol  efTccd  Vely  ctfimalo  the  val ue  ■  ■ !  :  i  ^  I « n  \  mg  vur  ion*  !  eve  U ,  And  the  D 
coimnujut}  cannot  effectively  describe  what  let  els  are  posable.  A*  a  raaulL, 
autonomy  by  itself  does  not.  provide  signirkani  assistance  in  choosing  priorities 
from  among  many  potentially  rdet  m  area--.  of  RAD.  The  objective  of  the  first 

. . .  of  this  Roadmap  it  to  cut  the  Gordian  khol  of  autonomous 

mobility. 

To  compound  ilie  problem,  mobility  h  not  the  only  UGV  system  ftuseton  that 
requires  more  autonomy.  Brief  deaeriplioiw  of  other  Kune  lions  are  provided 
below-  die  raided  needi  are  fully  dealt  with  in  die  re co m  men  d  ati  c  ns  of  the  Task 
Force. 

Au lon< >mou*  M ol ill li y  is  a  k e y  cipahhi y.  A 1  ihi>u^h  there  ar e  many 
opituuns  about  the  levels  of  auto® loitlv  that  ure  required  for  effective 
mobility,  ilieic  n.  little  argumeiii  ihn&igiufkanl  advances  .irt  needed  m 
die  abilities  of  UGVi  to  gel  from  owe  point  to  another  with  hide  or  no 
limnan  help  alarm  'he  wa*  If  UGV*  are  to  emerge  from  their  current 
situation  of  niche  applications.  additional  autonomy  of  mobility  is  critical. 
For  this  reason  a  aigmfkaiil  portion  of  lire  roadmap  is  concerned  with 
accdcraiiiig  development  ef  amonomtHis  mobility. 

Cwrrlinatd  Action  of  ‘Minion  package  And  I  GV,  T  oday  little  or  no 
research  is  focused  on  the  capabilities  necessary  to  coordinate  die  actions 
of  miation  packages  and  UGVh.  A*  an  cymplc  of  the  ncedH  an 
anionomou.dy  mobile  UGV  with  a  functjotul  Automatic  I  irirci 
Recognition  (AIR)  system  will  be  iiscltsi  if  the  UGV  is  unable  to  ensure 
good  Line'  of  sight  for  die  AT  R 

.  \  ii  lonomoiu  i  ’oope ra  I  km  A  mong  l  IGYsa,  lire  m  tail  on*  identi  fi  ed  for 
the  longer  term  will  requue  sclf'-generaled  cooperation  among  teams  of 
UGVk.  Tins  will  require  greatly  enhanced  communication,  and  deckicn- 
inaJang  capabilities.  for  example  1  he  iftformati an  sensed  by  ane  UGV  can 
b<  used  to  define  the  meltons  of  another  without  requiring  die  input  of  a 
remote  operator. 

Hie  bullets  above  call  fot  a  more  sy^iemalie  approach  lo  the  development  of 
ai]  lono  i  ny  of  di  ITcniig  type*  RasonanoeH  of  this  theme  will  be  Sic  aid  throughout 
the  re  1 1 1  uu  i  dei  of  ttie  document. 
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Downing  of  Iranian  Air  Lines  Flight  655  (IAL-655). 

On  July  3,  1988,  the  U.S.S.  Vincennes  shot  down  a  commercial  airliner  over  the  Persian 
Gulf.  These  circumstances,  combined  with  the  fact  that  an  Iranian  F-14  had  been  observed  taking 
off  from  an  Iranian  military  base  in  the  same  general  location  and  at  approximately  the  same  time 
as  the  commercial  airliner,  led  to  the  accident.  With  such  intense  expectations  of  an  attack,  and 
given  previous  observations  allowing  for  the  possibility  of  an  F-14  cloaking  itself  as  a  commercial 
airliner,  the  crew  was  falsely  led  to  believe  they  were  under  attack.  The  official  investigation  of 
the  incident  by  Admiral  Fogarty  used  the  phrase  "scenario  fulfillment"  to  describe  the  tactical 
information  coordinator’s  (TIC’s)  distortion  of  the  situation.  The  investigation  states,  "Stress, 
task  fixation,  and  unconscious  distortion  of  data  may  have  played  a  major  role  in  this  incident." 
(Based  on  Malone,  1990) 

The  accident  occurred  in  part  because  a  TIC  mistakenly  reported  the  aircraft  as  descending 
over  the  ship,  based  on  the  seaman's  perception  of  the  information  gathered  from  his  display, 
when  in  actuality  the  aircraft  was  ascending.  Two  other  crew  concurred  with  the  TIC's 
assessment  of  the  situation.  To  understand  why  this  error  happened,  we  must  examine  the  events 
leading  up  to  this  incident.  The  accident  is  a  classic  example  of  the  influence  of  expectancy  in 
causing  human  error.  Already  in  a  stressful  environment,  naval  personnel  in  the  Gulf  were  told  to 
expect  an  attack  from  the  Iranians  on  Independence  Day.  In  addition,  specific  warnings  were 
issued  concerning  possible  attacks  from  Iranian  F-14  Tomcats.  From  June  2  -  July  2,  1988,  there 
were  150  challenges  issued  by  warships  to  aircraft.  Of  these,  83  percent  were  to  Iranian  military 
aircraft  (7  of  which  were  F-14s),  while  only  1.3  percent  were  issued  to  commercial  aircraft. 
Moreover,  Iranian  F-14s  were  previously  observed  flying  commercial  traffic  routes  for  cover, 
emitting  radar  squawk  signals  of  commercial  aircraft  in  tandem  with  their  military  squawk. 

Industrial  Accident  at  Union  Carbide. 

On  December  2,  1984,  the  accidental  release  of  a  toxic  chemical  from  a  Union  Carbide 
subsidiary  in  Bhopal,  India,  killed  at  least  2,500  people  (official  count)  and  quite  possibly  as  many 
as  10,000.  In  addition,  over  150,000  people  (from  a  population  of  672,000  in  this  urban  area) 
sought  medical  attention  resulting  from  the  accident.  The  incident  occurred  at  night  when  the 
pressure  relief  valve  blew  out  on  a  tank  containing  45  tons  of  methyl  isocyanate  (MIC),  releasing 
its  contents  and  bathing  the  city  of  Bhopal  in  suffocating  fumes.  MIC  is  a  highly  volatile, 
unstable,  flammable  chemical  that  becomes  a  gas  at  approximately  100  degrees  F  and  is  highly 
reactive  with  water  at  any  temperature.  In  this  accident,  the  MIC  in  the  tank  became 
contaminated  with  water  when  the  connecting  lines  were  flushed  during  a  routine  maintenance 
operation.  Within  a  period  of  less  than  two  hours,  the  pressure  in  the  tank  had  risen  from  its 
normal  acceptable  level  of  between  2  and  25  psig  to  well  in  excess  of  55  psig,  and  the  temperature 
had  soared  to  at  least  392  degrees  F,  causing  the  relief  valve  to  open.  The  accident  was  attributed 
to  human  error  caused  by  carelessness,  poor  training,  and  improper  maintenance,  as  well  as  design 
shortcomings  in  the  control  room.  Control  room  instrumentation  supplied  no  record  of  previous 
values  for  important  system  parameters  such  as  tank  pressure  to  supply  a  historical  trace  for  an 
operator  who  was  taking  over  a  shift  from  another  (the  accident  occurred  within  a  few  hours  of 
shift  change).  The  upper  limit  on  the  displays  for  the  tank  temperature  and  pressure  gauges  was 
too  low  to  adequately  reflect  the  conditions  in  the  tank  (for  example,  the  pressure  gauge  topped 
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out  at  55  psig).  The  tank  temperature  display  was  not  even  located  in  the  control  room.  In 
Bhopal  after  the  accident,  Union  Carbide's  chairman,  Warren  M.  Anderson,  expressed  his  concern 
for  the  loss  of  life,  but  maintained  that  the  "safety  standards  in  the  U.S.  are  identical  to  those  in 
India....  Same  equipment,  same  design,  same  everything"  (Casey,  1993).  Unfortunately,  that  was 
part  of  the  problem.  The  designers  of  the  plant  did  not  anticipate  the  cultural  differences  between 
the  operators  in  India  and  their  American  counterparts.  Nor  did  they  take  into  consideration  the 
fact  that  requiring  the  Indian  operators  to  keep  their  logbooks  in  English,  a  second  language  for 
the  Hindi  personnel  impeded  the  transfer  of  information  between  operators.  Finally,  two  safety 
mechanisms  that  could  have  contained  the  accident  failed.  The  neutralizing  soda  scrubber  system 
was  grossly  inadequate;  it  was  designed  to  absorb  190  pounds  of  gas  per  hour  at  15  psig  and  95 
degrees  F,  but,  at  the  time  of  the  accident,  the  MIC  was  flowing  at  40,000  pounds  per  hour  at  a 
temperature  of  over  392  degrees  F.  The  flare  tower  that  could  have  harmlessly  burnt  off  the  gas 
was  not  in  operation.  (Based  on  Malone,  1990;  Casey,  1993) 

Lessons  Learned 

Effects  of  joint  range  of  motion  limits  on  strength. 

When  the  angle  of  a  fully  deflected  aircraft  rudder/brake  pedal  is  beyond  the  limit  of  ankle 
mobility,  the  pedal  will  seem  to  have  excessive  resistance.  In  addition,  this  will  prevent  the  pilot 
from  fully  utilizing  the  brakes  of  the  aircraft.  (McDaniel,  1998) 

Component  accessibility. 

Failure  to  design  aircraft  components  and  equipment  for  easy  accessibility  causes  excessive 
maintenance  time  for  repair  and  decreased  aircraft  availability.  (Department  of  the  Air  Force, 

1996) 

Cross-connected  hydraulic  lines. 

Inadvertent  cross-connection  of  hydraulic  lines  can  easily  occur  when  similar  fittings  are  used 
for  all  lines,  leading  to  extremely  high  human  and  equipment  cost.  (Department  of  the  Air  Force, 
1996) 

Command  ejection  option. 

Lack  of  a  command  ejection  option  in  a  multiple-ejection-seat  system  can  have  two  major 
negative  effects:  seats  can  collide  because  ejections  are  not  coordinated;  and  one  or  more  people 
may  be  left  within  the  vehicle  if  they  are  unable  to  eject  themselves.  Either  of  these  situations 
could  result  in  loss  of  life.  (Department  of  the  Air  Force,  1996) 

Cargo-loading  procedures. 

When  a  winch  operator  loading  cargo  aboard  an  aircraft  cannot  see  the  cargo  being  loaded 
and  does  not  have  communications  with  the  rest  of  the  loading  crew,  safety  is  adversely  impacted. 
(Department  of  the  Air  Force,  1996) 

Overhead  power  lines  and  tall  vehicles. 

It  is  important  to  identify  overhead  hazards  such  as  low  power  lines  when  planning  and 
organizing  work  involving  tall  vehicles.  Without  proper  identification  of  workplace  hazards,  the 
ability  to  minimize  exposure  to  and  protect  personnel  from  hazards  is  significantly  reduced.  (Dept, 
of  Energy,  1998) 
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Tactical  altitude  director  system  audio  alarms. 

When  a  low-altitude  warning  system  sounds  frequent  false  alarms,  pilots  become  accustomed 
to  the  audio  alarm  and  tend  to  ignore  it.  This  can  result  in  loss  of  the  aircraft  and  aircrew  through 
failure  to  respond  to  a  valid  alarm.  (Department  of  the  Air  Force,  1996) 

Life-raft  deployment. 

Life  rafts  not  properly  designed  for  quick  and  easy  deployment  can  result  in  death  to 
passengers  and  crew.  (Department  of  the  Air  Force,  1996) 

Human  error  in  aircraft  accidents. 

Studies  of  commercial  jet  aircraft  accidents  attribute  over  70  percent  of  accidents  to  crew 
error.  (BOEING,  1993) 

Benefits  from  HE 

Nuclear/Bioloqical/Chemical  Reconnaissance  Vehicle. 

Redesigning  interfaces  utilizing  subjective  workload  assessment,  this  system,  originally 
requiring  a  four-member  crew,  was  successfully  reduced  to  three  members.  As  a  part  of  crew  risk 
reduction,  HE  design  principles  enabled  incorporation  of  standoff  detection  capability,  thereby 
reducing  exposure  risk  to  crewmembers.  In  addition,  HE  involvement  in  overall  system  design 
integration  allowed  for  considerable  reductions  in  maintainability  costs.  (Booher,  1997) 

Forward  Area  Artillery  Resupply  Vehicle. 

During  the  beginning  stages  of  system  development,  subjective  workload  analysis  and 
computer  modeling  were  used  to  determine  optimal  vehicle  crew  size  for  maximum  mission 
performance.  During  the  T&E  phase  of  the  design  process,  task  analysis,  and  field  observation 
techniques  were  utilized  to  identify  critical  areas  of  interaction  between  warfighters,  equipment 
and  environment  during  battle  scenarios.  Throughout  the  course  of  the  project,  these  HE 
principles  targeted  key  issues  that  were  subsequently  resolved  by  the  design  team  leading  to  a 
more  effective  product.  (Booher,  1997) 

Transport  aircraft  redesign  to  accommodate  parachutists 

A  joint  Air  Force- Army  working  group  helped  redesign  the  fuselage  and  door  position  of  a 
large  transport  aircraft  using  HE  principles  to  improve  the  airflow  for  parachutes.  The  vehicle  is 
now  considered  the  best  aircraft  for  parachute  jumping.  (Air  Force  HSI  Office,  1995) 

Screen  display  redesign. 

The  CRT  screen  display  used  by  the  directory  assistants  at  a  regional  telephone  company  was 
reconfigured  in  keeping  with  HE  recommendations.  After  the  redesign,  the  average  time  to 
process  a  call  dropped  by  600  milliseconds.  This  reduction  saves  the  company  $2.94  million  a 
year  across  the  five-state  region  served  by  the  system.  (Hendrick,  1996) 

Training  system  redesign. 

At  the  same  telephone  company,  HE  was  applied  in  redesigning  the  systems  used  to  train 
directory  assistants.  These  revisions  reduced  the  time  needed  to  train  an  operator  from  five  days 
to  one  and  a  half  days.  (Hendrick,  1996) 

Shoulder-launched  missile  system. 

A  shoulder-launched  missile  system  with  challenging  operational  requirements  received 
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extensive  human  engineering  during  its  concept,  design,  test,  and  fielding,  closely  following  the 
tasking  in  Section  4.  Not  only  did  it  meet  operational  requirements,  but  also  the  design  of  the 
user-system  interface  and  operational  procedures  enabled  what  might  have  otherwise  been  a 
complicated  system,  to  facilitate  effortless  field  use  by  third-world  freedom  fighters.  Its  original 
critics  now  praise  the  system  for  its  simplicity.  (Reed,  1988) 

Gunship  aft-scanner  workstation  redesign. 

Because  gunship  aft-scanners  (who  look  for  missile  launches  and  other  threats  from  the  tail 
of  their  aircraft)  had  to  remain  in  an  "unnatural"  prone  position  for  hours  over  hostile  territory, 
they  suffered  back  and  neck  pain.  To  remedy  this  problem,  HE  practitioners  identified  alternative 
design  solutions  for  the  aft-scanner  workstation.  The  program  office  was  able  to  fund  several  of 
these  recommendations.  As  a  result  of  the  HE  effort,  weight  distribution,  body  posture, 
positioning  for  mission  tasks,  functional  reach,  and  external  visibility  were  all  improved,  and  neck 
and  back  pain  complaints  declined.  (Gentner,  Wourms,  Hunn,  &  Farrell,  1996) 

Development  of  an  efficient  helicopter  tool  kit. 

A  striking  example  of  the  benefits  of  HE  involvement  is  the  design  of  the  tool  kit  for 
helicopter  mechanics.  Historically,  aircraft  maintenance  tool  kits  have  been  large,  awkward,  and 
heavy  to  transport  on  deployments.  Based  on  HE  recommendations,  the  organizational  tool  kit 
for  one  helicopter  was  reduced  from  134  tools  to  only  6  tools.  This  redesign  reduced  what  is 
usually  a  trunk-sized  kit  to  a  canvas  pouch  that  is  approximately  half  the  size  of  a  rolled-up 
newspaper.  (Skelton,  1997) 

Antisubmarine  Warfare  System. 

Using  mission  task  and  function  analysis  methods  the  HE  practitioner  shaped  the  design 
process  of  this  system.  The  designers  were  able  to  meet  mission  objectives  while  incorporating 
many  off-the-shelf  components,  lowering  overall  system  cost.  During  T&E,  the  system 
substantially  exceeded  customer  expectations,  and  subsequently  the  design  lead  to  a  highly 
successful  deployment.  (J.  B.  Shafer,  personal  communication,  May  7,  1998) 

Experimental  Helicopter  Technological  Integration. 

During  mid-nineteen  eighties  studies  were  conducted  to  determine  if  a  single  pilot  could 
perform  a  scout/attack  mission  using  advanced  cockpit  technology  that  previously  required  two 
aviators.  Function  analysis  was  used  to  determine  which  mission  segments  had  the  greatest 
workload,  and  pilots  were  interviewed  regarding  flight  deck  automation  and  workloads.  Next, 
pilots  were  assessed  using  fixed  based  simulators  installed  with  a  prototype  cockpit  design.  The 
tests  and  interviews  led  to  two  conclusions:  (1)  a  highly  effective  pilot  orientated  cockpit  was 
designed,  and  (2)  although  an  exceptional  single  pilot  could  perform  the  scout/attack  mission, 
under  battle  stress  the  pilot  would  become  overloaded  and,  consequently,  the  mission 
performance  would  be  sacrificed.  Due  to  early  HE  involvement,  the  military  had  the  opportunity 
to  discontinue  a  project  before  any  further  investment  was  required.  (J.  B.  Shafer,  personal 
communication,  May  7,  1998) 
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